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THE CONDENSATION PHASE TRANSITION IN THE REGULAR fc-SAT MODEL 


VICTOR BAPST* *, AMIN COJA-OGHLAN* 


Abstract. Much of the recent work on phase transitions in random discrete structures has been inspired by ingenious 
but non-rigorous approaches from physics. The physics predictions typically come in the form of distributional fixed 
point problems that are intended to mimic Belief Propagation, a message passing algorithm. In this paper we propose 
a novel method for harnessing Belief Propagation direcdy to obtain a rigorous proof of such a prediction, namely the 
existence and location of a condensation phase transition in the random regular fc-SAT model. 
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1. Introduction 

1.1. Background and motivation. Over the past three decades the study of random constraint satisfaction prob¬ 
lems has been driven by ideas from statistical physics |22l|23]. This work has had a substantial impact on com¬ 
puter science (e.g., proofs that certain benchmark instances are difficult for certain algorithms), coding theory 
(“low density parity check codes”) and probabilistic combinatorics (random graphs, hypergraphs and formulas); 
e.g., O [l^ Uni [TT] [181 [19] |28) . All of these disciplines deal with a common setup. There are a large number of 
“variables” that interact through a similarly large number of “constraints”. Each variable ranges over a finite do¬ 
main (such as the Boolean values ‘true’ and ‘false’) and every constraint binds a small number of variables, either 
encouraging or discouraging certain value combinations. 

The striking feature of the physics work is that it is based on a non-rigorous but generic approach called the 
cavity method, centered around the Belief Propagation message-passing algorithm, that can be applied almost 
mechanically [21] . Hence the impact of a single technique on such a wide range of problems. By comparison, the 
rigorous study of random problems has largely been case-by-case. This begs the question of whether the Belief 
Propagation calculations can be put on a rigorous basis more directly. 

This is precisely the thrust of the present paper. We show how the physics calculations can be turned into a rigor¬ 
ous proof in a highly non-trivial and somewhat representative case. Specifically, we determine the “condensation 
phase transition” in the random regular fc-SAT model. The proof is based on a novel approach that demonstrates 
how our recent general results on the connection between spatial mixing properties and the computation of the 
free energy [S] can be put to work. The centrepiece of the proof is a fairly direct analysis of the Gibbs marginals by 
means of Belief Propagation. The arguments are rather generic and we expect them to extend to other problems. 

The random regular fc-SAT model is defined as follows [2^. There are Boolean variables xi,...,x„ and m con- 
traints, namely propositional clauses of length fc. Each variable occurs precisely dl2 times as a positive and pre¬ 
cisely d!2 times as a negative literal. Hence, m = dnl[2k]; we assume tacitly that d is even and that fc divides 
dn. Let <1> = fc(u) signify a uniformly random such fc-SAT formulaQ For fc exceeding a certain constant fco the 
threshold where O ceases to be satisfiable is known I91.0 While the exact formula is cumbersome, asymptotically 
dk-SAj/k- 2*^ln2- fcln 2 / 2 - 1 - 0(1) for large fc. 

Of course, finding the satisfiability threshold is hardly the end of the story. Much more precise information 
is encoded in the Hamiltonian a ^ B^(a] that maps each truth assignment a to the number of clauses that it 
violates. We think of it as a “landscape” on the Hamming cube. For instance, if E® is riddled with local minima, we 
should expect that Markov processes such as Simulated Annealing get trapped [T||20||2^ . Hence, E® holds the key 
to understanding algorithms for finding, counting and sampling solutions [25l|27] . 
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The key quantity upon which the study of the Hamiltonian hinges is the partition function 

Z(D : jSe (0,oo) ^expC-jSiioCo')). 
a 

As usual, the larger the inverse temperature f, the bigger the relative contribution of “good” assignments that vio¬ 
late few clauses. Of course, we are interested in the asymptotics as n oo. Since Zq>{p) scales exponentially with 
n, we consider 

(pd.k-P^iO.oo)^ lim -E[lnZ<i,()S)]. (1.1) 

n—oo n 

Clearly, what makes (pd.k vicious is that the log is inside the expectation. The existence of the limit follows from the 
interpolation method (8] andAzuma’sinequality implies that In ZoC/i) concentrates about E [In ZmC/l)]. 

A key question is how smoothly ipd.k^P) varies as a function of f for fixed d, k. Formally, let us call e (0,oo) 
smooth if there exists £ > 0 such that the function /3 e (/Iq - £, /Iq + £) '-^ <Pd,k admits an expansion as an absolutely 
convergent power series around /Iq. If fa fails to be smooth, a phase transition occurs at fa- 

1.2. Results. According to (non-rigorous) physics predictions [20] for certain values of d close to the satisfiability 
threshold d^-sAT there occurs a so-called condensation phase transition at a certain critical ;6cond(^^> ^1 > 0- The 
main result of this paper proves this conjecture. Let us postpone the precise definition of /lcond(t^) fc) for a moment. 

Theorem 1.1. There exists fco - 3 such that for all k > ko, d < d^-sAT there is f candid, k) e (0,oo] such that any 
fe iO, f candid, k)) issmooth. If f candid,k] <oo, then there occurs a phase transition at f candid,k). 


Thus, if we fix d,k such that f candid, k) - oo, then the function (pd,k is analytic on (0,oo). But if d,k are such 
that f candid, k) < oo, then (pd,k is non-analytic at the point f candid, k). In fact, we will see that f candid, k) < oo 
for d exceeding a specific dcond(fc) < d/c-SAT- Crucially, Theorem ll.ll identifies the precise condensation threshold 
/3cond(ri> k); it is the first such result in a model of this kind. 

Let us take a look at the precise value of /Icond id,k]. As most predictions based on the cavity method, fcand id, k) 
results from a distributional fixed point problem, i.e., a fixed point problem on the space of probability measures 
on the unit interval (0,1). The fixed point problem derives mechanically from the “IRSB cavity equations” \2J\ . 
Specifically, writing 13*(0) for the set of probability measures on Q, we define two maps 

l^k.d.p : 159(0,1) - ^?>(0,1), ^k.d.p : 159(0,1) - ^39(o, i) 

as follows. Given ti e i) let rj - (rji,... ,rik-i) £ (0,1)*^“^ be a random k- 1-tuple drawn from the distribution 
izirj)lZ{Tt)) d0^“J ;r(? 7 y), where 

z(77) = 2-(l-exp(-;6)) P Py and Z[7t) = f z[Tj)d(^ 

j<k j<k 

Then (7t) is the distribution of (l-(l-exp (-/))) Yl^I^pflziri]. Similarly, given if e ^?>(0, 1) draw?) = 
from (z(p)/Z(7f))d0^“J ftifij), where 

z(t7)= n hi n n i^~hj'> n hj> Z(7f)= f ziTj]d(^7iifjj]. 

j<dl2 J>dl2 J<d/2 i>dl2 J<k 

Then ^k.d.pi^) is the distribution of (nj<d/2i7j Y\j>di 2 il ~ hj))!zijf). Call a distribution n e 1) skewed if the 
probability mass of the interval (0,1 - exp(-fc/3/2)) satisfies 7t(0,1 - exp(-fc/3/2)) < 

Proposition 1.2. Letd-(k] = d^- SAJ-k^ and f-{k,d] = A:ln2-101nk. The map’^^^.p - ^k,d,p°^k,d,p has a unique 
skewed fixed point tt'^ ^ provided that k> ko, de [d- (fc), dfc-sAx] tind f>f- ik, d). 


To extract f candid, k), let vi,...,Vfc,vi,...,Vd be independent random variables such that the v; have distribu¬ 
tion ^ ^ and the v,- have distribution I^k,d,^itt \^ Setting 

^ 1 = n n ( 1 -^;)+ n n 

j<d/2 j>dl2 j<dl2 J>dl2 

and Z 3 = vivi -i- (1 - vi)(l - vi), we let 


Z 2 = 1 - (1 - exp(-;6)) Y[ Vj 
j^k 


^ d „ Ejzilnzi] dE[z2lnz2] Eizalnza] 

,^(A:,d,; 6 ) = lnE[zi] + -lnE[z2]-dlnE[z3], SS{k,d,f)^ ^ d 

k E[zi] k E[z 2 ] E[z 3 ] 


( 1 . 2 ) 
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Finally, with the usual convention that inf 0 = oo we let 


/^cond(^> d.) 


loo if d<d-(k], 

|inf{/5> l3-{k,d) :^{k,d,p) <SS{k,d,j3)] ifde [d-{k),dk-SNi\- 


We proceed to highlight a few consequences of Theorem 11.11 and its proof. The following result shows that 
Pcondidyk) < oo, i.e., that a condensation phase transition occurs, for degrees d strictly helow the satisfiahility 
threshold. 


Corollary 1.3. Ifk > fco, then <icond(fc) = min{d > 0 : )6cond(^^. k) < oo} < - D.{k). 

Furthermore, the following corollary shows that the so-called “replica symmetric solution” predicted hy the cavity 
method yields the correct value of (pd,kiP) for (} < /3cond(^> k). 

Corollary 1.4. Ifk> ko, d < d^sAT andp< Pcondid>k), thencpd^kiP) - t^{k,d,p). 

Corollary [L4] opens the door to studying the “landscape” Eq, for /3 < (fcondid,k). Specifically, Corollary [T]4] en¬ 
ables us to bring the “planting trick” from [1] to bear so that we can analyse typical properties of samples from the 
Gibbs measure. We leave a detailed discussion to future work. Finally, complementing Corollarv ll.4l the following 
result shows that ^ik,d,l3) overshoots (pd,k{p)iox (i> Pcondid,k). 

Corollary 1.5. If k > ko, d < d^sAT tind f > fcondld.k), then there is fcoad^d,k) < f' < f such that tpd,kiP') < 
^[k,d,p'). 

1.3. Outline and related work. Admittedly, the definition of fcQadik,d) is not exactly simple. For instance, even 
though the fixed point distribution from Prooosition ll.2l stems from a discrete problem, it turns out to be a contin¬ 
uous distribution on (0,1). Yet perhaps despite appearances, the analytic formula 11.21 is conceptually/ar simpler 
than the definition of ipd,k- For instance, we are going to see in Section|2]that the fixed point problem can be under¬ 
stood elegantly in terms of a Galton-Watson tree. Thus, one could say that Theorem I 1.1 [ reduces the condensation 
problem on the complex random formula fl> to a problem on a random tree. 

The proof of Theorem 1 1.1 1 builds upon an abstract result from 0 that, roughly speaking, reduces the study 
of the partition function to two tasks. First, to calculate the marginals of the Gibbs measure induced by a random 
formula ® chosen from a reweighted probability distribution, the “planted model”. Second, to prove that the Gibbs 
measure of <l> enjoys the non-reconstruction property, a spatial mixing property. The technical contribution of the 
present work is to actually tackle these two tasks problems in a fairly generic way. Our principal tool is going to 
be the Belief Propagation algorithm, the cornerstone of the physicsts’ cavity method. In particular, we are going 
to reduce the see that the distributional operator '^k.d.p from Proposition [O] mimics Belief Propagation run on 
a Galton-Watson tree that captures the local geometry of the formula ®. The predictions of the “cavity method” 
typically come as distributional fixed points but there are few proofs that establish such predictions rigorously. 
The one most closely related to the present work is the paper of Bapst et al. (6) on condensation in random graph 
coloring. It determines the critical average degree d for which condensation starts to occur with respect to the 
number of proper fc-colorings of the Erdos-Renyi random graph. Conceptually, this corresponds to taking the 
limit jS ^ oo in 11.11 . which simplifies the problem rather substantially. Thus, the main result of corresponds 
to Corollarv ll.3l Other previous results on condensation, which dealt with random hypergraph 2-coloring and the 
Potts model on the random graph, were only approximate [7l [TT][T2l . 

Interestingly, determining the satisfiability threshold on the random regular formula ® is conceptually much 
easier than identifying the condensation threshold |9]. This is because the local structure of the random formula 
® is essentially deterministic, namely a tree comprising of clauses and variables in which every variable appears 
d/2 times positively and d/2 times negatively. In effect, the satisfiability threshold is given by a fixed point problem 
on the unit interval rather than on the space of probability measures on the unit interval. Similar simplifications 
occur in other regular models [14] [T^, and these proofs employed Belief Propagation in this simpler setting. By 
contrast, we will see in Section |2] that the condensation phase transition hinges on the reweighted distribution ®, 
whose local structure is genuinely random. 

Recent work on the fc-SAT threshold in uniformly random formulas |9l[T0] , in particular the breakthrough p^er 
by Ding, Sly and Sun [16] , also harnessed the physicists’ Belief Propagation or Survey Propagation calculations^ In 
the uniformly random model a substantial technical difficulty is posed by the presence of variables of exceptionally 


^Survey Propagation can be viewed as a Belief Propagation applied to a modified constraint satisfaction problem En. 
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high degree, an issue that is, of course, absent in the regular model. Specifically, O [TOl [IH apply the second 
moment method to a random variable whose construction is guided by Belief/Survey Propagation. By contrast, 
here we employ Belief Propagation in the more direct way enabled by |5] . 

1.4. Notation and preliminaries. We generally view a regular fc-SAT instance <1> as bijections from sets of clause 
clones to sets of variable clones (“configuration model”). That is, given n, m, d, k, we let {xi,..., x,|} x [d] be the set 
of variable clones and {ai,..., am] x [k] the set of clause clones. Then <1>; {xi,..., x„} x [d] [ai,..., am] x [k] is a 

bijection. The first d/2 clones of each variable are considered its positive occurrences and the last d/2 ones its 
negative occurrences. 

We denote the image of a clone {Xi,j) by d^{Xi,j) and the inverse image of [ai,]) by d^{ai,j). Analogously, 
is the depth-^ neighborhood of clone {v,j). Moreover, we define fl> as a uniformly random bijection. By 
standard arguments this distribution is easily seen to be contiguous to the uniform distribution on regular formu¬ 
las. 

Suppose that the variables and clauses of (P,®' are x;,x'.,a/,a', for i e [n], j e [m]. We distinguish (variable 

I J J 

or clause) clones r, r' of <P,®', which we consider their roots. An isomorphism ^ is a bijection with the 
following properties. 

ISMl: r'^y/[r). 

ISM2: y/ maps variable clones to variable clones and clause clones to clause clones. 

ISMS: lfy/{v,h]-{w,j],thenh-j. 

ISM4: We have i/a o <p(y, h) = <P' o (i>, h) for all clones [v,h). 

Let ^ > 0 and let T be a regular fc-SAT formula with a distinguished (variable or clause) clone r. For each variable 
clone (x, i) of <1> we have a random variable l{d^T = d^(x, /)} that indicates that the depth-f' neighborhood of ® 
rooted at (x,/) is isomorphic to T. Similarly, for each clause cone {a,j] of ® we consider the random variable 
I 0 f+i J = d^^(a,]']]. Let be the cr-algebra generated by all these random variables. Thus, captures the 
“local structure” of the random formula up to depth £. 

2. Outline 

2.1. Two moments do not suffice. The default approach to studying the function (pd,kiP) is the venerable “second 
moment method”. Cast on a logarithmic scale, if 

1 o 2 

limsup — lnE[Z(i>(/3)'^] < lim — lnE[Z®(/3)], then (2.1) 

n—<x 3 n n^co n 

(pd.kiP)^ lim -lnE[Z®()S)]. (2.2) 

n—oo fi 

The last term is easy to study because the log is outside the expectation. In particular, the function (i e (0, oo) 
lim„^oo -^InElZ®!/!)] turns out to be analytic. Consequently, the least (i e (0,oo) where <2.21 fails to hold must be a 
phase transition. 

From a bird’s eye view, both the physics intuition and the second moment are all about the geometry of the 
Gibbs measure of ® at a given /3 e (0,oo). Let us encode truth assignments as points cr e {+1}” with the convention 
that 1 stands for ‘true’ and -1 for ‘false’. Then the Gibbs measure is the distribution on {+!}” defined by 

(7 e {+!}" -- exp{-l3E^{a))/Z^il3). 

Thus, we weigh assignments according to the number of clauses that they violate, giving greater weight to ‘bet¬ 
ter’ assignments as /3 gets larger. Let <t,o’i,<T 2 ,... be independent samples from the Gibbs measure and write 
{X{(Ti,...,ai))(p p for the expectation of X : ({+1}”)^ ^ IR. Then according to the physics picture the condensation 
point /icond(fc) should be the supremum of all /5 > 0 such that E(|<ti • (T 2 \)<b,p - o[n). In other words, if we choose a 
random formula ® and then sample two assignments <ti , <T 2 according to the Gibbs measure independently, then 
<ti,<T 2 will be about orthogonal. This decorrelation property is, roughly speaking, a necessary condition for the 
success of the second moment method as well (2|4]. Therefore, the prediction that Edcri •<T 2 |>(i>j 3 = o{n) right up 
to Pcondid,k] may inspire confidence that the same is true of 12.11 . In fact, we will prove in Section [7] that <2.11 
holds if either d or ^6 is relatively small. 

Lemma 2.1. Ifd < d- (fc) or p< fi- (fc, d) then (2.1\l is true. 
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However, for p near Pcondid,k) the second moment method turns out to fail rather spectacularly. Formally, if 
/3cond(<^> k) < oo, then there exists £ > 0 such that I I2.H is violated for all p > /3cond(<^> k) - e, i.e., the second moment 
overshoots the square of the first moment hy a factor that is exponential in n. 

2.2. Quenching the average. To understand what goes awry it is convenient to turn the second moment into 
a first moment under a reweighted distribution that we call the planted model. This is the distribution on for¬ 
mula/assignment pairs under which the probability of equals exp{-pE^id))lElZ(p{P)]. Let (O.d") be a ran¬ 
dom pair drawn from this distribution. Then by symmetry the distribution of the assignment a is uniform and 
we may assume without loss that o’ = 1 is the all-ones assignment. Further, the probability that a specific formula 
® comes up equals IP[® = <!)] = Zp{^)lE[Zp{Q>]]. Thus, the planted distribution weighs formulas by their partition 
function. In effect, 

E\Z^{pf]^E[Z^[P)]-E[Z^[P)]. 

If we go over the proof of Lemma [ZTl we see that ElZ^ip)] is dominated by two distinct contributions. First, 
assignments that are more or less orthogonal to & yield a term of order E[Z®(/1)]. Second, there is a contribution 
from <T close to d = 1; say, <7 • 1 > n{\ - Geometrically, this reflects the fact that the planted assignment 1 

sits in a “valley” of the Hamiltonian w.h.p. The valleys are known as clusters in the physics literature and we let 

Z^(P){\{CT ■!> n{\-2-^'^^)})^ f, 

be the (weighted) cluster size. Performing an elementary calculation, we find that it is the expected cluster size that 
derails the second moment method for p near Pcondid, k). 

At a second glance, this is unsurprising. For ^ (/5) scales exponentially with n and is therefore prone to large 
deviations effects. To suppress these we ought to investigate EllnSg'^ ^(/3)] instead of El*^^ ^(^)]. A similar issue 
(that the expected cluster size drives up the second moment) occurred in earlier work on condensation [6l[7l|TTl[T3 . 
Borrowing the remedy suggested in these papers, we prove in Section[7]that applying the second moment method 
to a carefully truncated random variable yields 

Lemma 2.2. Equation 12.21 ) holds iff 

limsupn'^EllnS^A^ld)] < lim n“^lnE[ZA(d)]. (2.3) 

n—*oo ' n—*oo 

Computing lim„^oo lnE[Z^(/i)] is easy, as the following standard lemma shows. 

Lemma 2.3. Assumethatd< djcSATtmd peU. Then\imfi^^n~^hiE[Z^{P)] - t^[k,d, P). 

Hence, we are left to calculate Elln';^^ ^(;6)], the “quenched average” in physics jargon. As the log and the ex¬ 
pectation do not commute, this problem is well beyond the reach of elementary methods. Tackling it is the main 
achievement of this paper. Specifically, we are going to prove 

Proposition 2.4. Assumethatde [d-{k),dic-SAT] andp> P-{k,d). Thenlimn^ca n~^E[ln‘^^ ^iP)] - SS{k,d,P). 

2.3. Non-reconstruction and the Bethe free energy. In the following we let for a formula $ and v e V u E and 
^ > 0, j; (resp. v] denote the set of vertices at distance exactly £ (resp. less than £] from v in ®. 

To prove Prooosition l2.4l we investigate the spatial mixing properties of the conditional Gibbs measure 

Specifically, for a variable x, an assignment a e and an integer ^ > 0 let V(<f),x, ^) be the cr-algebra on 

generated by the random variables er{y] for variable y at distance greater either ^ or ^ -i-1 from x. Further, 
we define 

Mf^(±l)= [l{«'(^) = ±lHV(®,x,al4^^((f). (2.4) 

In words, (+1) is the probability that x gets assigned +1 in a random assignment of its depth-^ neighborhood 
under the boundary condition induced by a. 

We lift the distributions from 12.41 to clauses. In slightly greater generality, suppose that p is a map that assigns 
each variable x a probability distribution px £ ^3*({±1}). Then for clause a of a formula <5 we let be the 

distribution on {±1}*^ with the following two properties. 

(i) if j E [fc] and x = d^{a,j), then the marginal distribution of the jth coordinate coincides with px- 
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(ii) subject to (i), + (Ini//a) (ac+ij is maximum. 

^j>,a 

These two conditions determine uniquely (because the entropy is concave). 

Now, we say that a formula O has the non-reconstruction property if for any e > 0 there is ^ > 0 such that 


lim P 

n—*oo 




< £ 


= 1 


The first half of the proof of Proposition l2.4l consist in proving the following. 


Proposition 2.5. Assume that d e [d-{k),dic-sAT\ and(i> P-{k,d). ThenO has the non-reconstruction property. 


Together with results from |5] Proposition l2.5l imDlies an upper hound on Efln"^^ ^ (/!)]. But to obtain a match¬ 
ing lower hound a little more work is needed. Specifically, we need to consider a further distribution on for¬ 
mula/assignment pairs that we call the planted replica model (fl>, a] generated by the following experiment. 

PRl: Choose a random formula ®. 

PR2: For each variable x choose dr(x) from independently. 

PR3: For every clause a choose aia] e {+1}^ independently from 

PR4: Choose ® uniformly at random subject to the following conditions. 

• If {a,j) is a clause clone and d^[a,j) - {x, i), then a[a,j) - dr(x). 

• For all clause clones {a,j) we have A^'''^{a,j) = A¥'*'^(a, j). 

If no such ® exists, start over from PR2. 

The planted replica model has the non-reconstruction property if for any £ > 0 there is ^ > 0 such that 


lim I 

n—*oo 




< £ 


= 1 


Proposition 2.6. Assume that d e [d- [k),dic-sAT\ tind P > P- [k, d). Then the planted replica model has the non¬ 
reconstruction property. 


The non-reconstruction property enables us to determine E[lnS#’^^(/3)]. Indeed, given a map p : x ^ px ^ 
15*({+1}) that assigns each variable a distribution on +1 we define the Bethefree energy of a formula ® as 

SS^(.p)^Y.^l-d)H[px)-i-Y,\H{a) +{In'll/a) ■ 

Of course, pa is the extension of p from variables to clauses as defined above. We also let g - t^^((p?^^]xev)- 
Then by combining Propositions l2.5l and l2.6l with (5] Theorems 4.4 and 4.5] we obtain the following. 

Corollary2.7. We havelim„^oo ■^Elln'i^^, ^(;6)] = lim^^oolimn-oo 

Furthermore, g is determined by the local structure of ® . Since O, the local structure of the random formula 
can be described in terms of a random tree. In fact, tracing the Belief Propagation algorithm on this random tree 
enables us to relate to the distributional fixed point problem from Proposition 11.21 The result of this, 

derived in SectionlH is 

Proposition2.8. We haveiimg^oo^imn^oo g] = SS{k,d,p]. 

Finally, Proposition l2.4l follows from Corollarv l2.7l and Proposition l2.8l 


3. Belief Propagation on random trees 

In the following of the paper we assume thatd e [d-ik),dic-sAT\ tind that p > P-{k,d]. We letcp = 1 - exp(-/3) e 

( 0 , 1 ). 

In this section, we introduce a Galton-Watson process on trees, that will describe the local neighborhood of 
randomly chosen vertices in random formulas but also allow to analyze the probabilistic fixed point problem in 
SectionlH 


6 






3.1. A Galton-Watson process on trees. We consider the following Galton-Watson process. We first observe that 
there is a unique q - q{k,d, p) e {0,1) such that 

l-(l-exp(-;S))^7*^ = 211-^7). (3.1) 


We start from the tree T 2 e of depth 2£ {£ > 0) such that 

• Each node at an even depth of the tree is a variable node and has for offspring d-\ clause nodes. 

• Each node at an odd depth of the tree is a clause node and has for offspring fc - 1 variable nodes. 

Let V 2 { be the set of T 2 e variable nodes, F 2 ( be the set of its clause nodes. Let d 14/ denote the subset of variables at 
distance 2 ( from the root, and for node v of T 2 e, let dv (resp. dj v) denote the set of neighbors (resp. children) of v 
in T 2 e. We further decorate T 2 ( has follows. Each node v e 14^ U14/ carries a number bv,\ e {-1,1} determined by 
the following process. 

(i) For the root r, we have br,] - 1 with probability q and br,\ = -1 with probability \ - q. 

(ii) The offspring of a variable node x with bx,\ = +1 is | - 1 clause nodes a with ba,\ = +1 and ^ clause nodes 
a such that + 1. 

(iii) If a clause node a is such that ba,\ = -1, the number of x e dj a such that = -1 has distribution Bin(fc - 

L1-^7)- 

(iv) If a clause node a is such that ba,\ - 1, then with probability expi-p)q^~^!{\ - (1 - ex^{-p)q^~^) the 
offspring is A; - 1 variables x with bx,\ - 1, and otherwise the number of x e dj a such that bx,\ = -1 has a 
conditionnal distribution distribution Bin>i(fc- 1,1 - < 7 ). . 


Then we define for a clause a and x e da, ba,x - bx,\ if x e dja and ba,x - ba,\ otherwise. We let d+ia = {x e 
da, ba,x = +1} and for a variable x, d+ix = {ae dx, ba,x = +!}■ We finally let, for 0 < / < fc, d+i ;x = {a e d+ix, |{y e 

da\{x},ha,y = l}| = /}. 

Let T = T{d,k,P2() be the resulting random (decorated) tree, let denote its distribution, and let 54/ 

denote the support of Similarly, we denote by T the random tree pending below the first clause adjacent to 

the root of T{d, k,p,2( + 2), by V 2/+1 its set of variables and by dV 2 /+i its set of variable at distance 2/ + 1 from the 
root, by its distribution, and by 52/+i the support of 

We call a sequence dv e &>{{-!, ^ boundary condition over T e 54/. Similarly, for T e 52/+i, we call a 

sequence dv e ^^({-1,1})'^'^^'’+^ a boundary condition on T. 

We define the Belief Propagation messages induced by the boundary condition dv on T as the families (v^ ^ ) 
and (v^’j'')aEf 2 ^> where = {dv)x for x e dV 2 /, and otherwise 


T,dv, , 








for X E 14/ \ 5^4/ ^ttd s E {-1,1}, 


^Sae(-l,l)3'2 ^Sx=s1f'a,p iSa) Ilyei?! a t'yf ^ 


Va,l5 (Sa) Ilyeaj a 't'yj (■^y) 


for a E F 2 / and s e {-1,1}. 


(3.2) 

(3.3) 


In the following of this section, we let ( large enough be fixed. We will be interested in showing that, under 
reasonnable assumptions, the message exiting a tree T e 54/ only weakly depends on the boundary conditon dv. 
More precisely, we define dv^”^ e 5*({-1, l})^^^f by dv®'(l) = 1=1- dv^^^-l) for all x e dV 2 /- For T e 54/+i we 
define dv^'’^ e ^{{-1, 1})'^^ similarly. For a tree T e 54/ with root r, and a boundary condition dv on T, we denote 
by 

^dv _ zdv ( 2 /) _ 

Vt - ''r,\ ’ ''t ~ *^r,t 

Similarly for T e 54/+1 with root r and a boundary condition dv on T we let 

9<5v _ -r,(5v -(2/+1) _ -r.av™ 

We are now ready to state the main results of this section. In the following, we denote by T a random tree drawn 
from the distribution pf^J^, and by dv a random boundary condition, independent of T and that satisfies the 
following condition. 

H ForanyxEdl4/, P [(dv);c(l) < 1 - exp(-fc;6/2) |((dv)y)^^g^,^^^j^j ] < 




Similarly, we denote by T a random tree drawn from the distribution , and by dv a random boundary con¬ 

dition, independent of T and that satisfies the following condition. 

H ForanyxEdf^ 2 t'+i, ^ < 1 - exp(-fcj8/2) ] <2-°-^^ 

Proposition 3.1. We have 

p[iivf.’'-v'2^>iioc>2ri] <r\ 
p [WVf'-vf^'^Woo > 2fc2exp(2;6)ri] < k^r\ 

The following variant of the proposition will follow from similar steps, and will prove usefull when analyzing 
random graphs in Sec[5l[6l We first need to slightly generalize the process considered up to now. Let GW{k, d, (i, 2() 
denote the Galton-Watson process introduced considered up to now. Let GW' (fc, d, /3, 2() he the multi-type random 
process defined hy the same rules as GW[k,d,p,2£), except for the first and second rule which are replaced by 

(i) ’ The root r has for offspring ^ clauses nodes a with ba,] - 1 and ^ clauses nodes a with ba,i = -1. 

(ii) ’ The offspring of a variable node x dijferentfrom the root with j = +1 is ^ -1 clause nodes a with j = +1 

and j clause nodes a such that ba,] + I- 

Let 32 ( denote the set of trees generated by the process and hy ^ the associated probability distribution. Let, 
in the following, T' denote a random tree drawn from this distribution. Let denote the variables ar distance 2£ 
from T' root (which is, as previously, a deterministic quantity). Let us call as before dv e &*{{-!, a boundary 

condition over T', and let us extend condition H into 

H’ ForanyxEdV^'^, P [(dv);c(l) < 1 - exp(-fcj8/2) 

Proposition 3.2. Assume that dv" is a random boundary condition over dV^^, independent of T' and that satisfies 
H’. the following assumption. Further assume that the random boundary condition dv' (whose distribution may 
also depend on T') satisfies for all x e dV' 2 ^, dv' xW > dv"(l). Then 

P [llp^y' > fcexp(;6)^-i] < r\ 

3.2. The (random) trunk of random trees: proof of Proposition lsTl In order to prove Proposition l3.ll we shall 
identify a concrete condition on {T,dv) under which the message is close to We define the Trunk of T 
under the boundary condition dv, Trunk(r,dv), as the largest subset W of 14/ such that for any xeW either 
TRO: X E dV 2 ^ and dvxil) > 1 - exp(-fc/J/2) 
or the five following conditions hold 

TRl: there are at least [O.SfcJ clauses a e dj x such that di a = {x}. 

TR2: there are no more than fO.lfcl clauses aedx such that |d_ial = k. 

TR3: for any 1< I < k the number of a e d_ix such that Idi^l = / is bounded by k^^^ll\ . 

TR4: there are no more than clauses a e dix such that |dial = 1 but da <f W. 

TR5: there are no more than clauses a e d_ix such that |d_ia| < k and |dia \ W| > |dia|/4. 

We will first observe that the following is true. 

Lemma 3.3. We have 

P [the root ofT under the boundary condition dv is cold] > 1 - 

Proof. The lemma is easily proved by induction. □ 

Further, we need to introduce the following definitions. For T e 32e with root r and x e dl^ 2 t'> we denote by 
[x ^ r] the unique shortest path from x to r in T. 

(i) We say that a factor node a e p 2 e is cold if and only if di a n Trunk(r, dv) 0. 

(ii) We say that a variable node x e V 2 f is cold if x e Trunk( T, dv). 

(hi) We say that (x, a) (with x e dj a) is cold if x is cold or a is cold. 

(iv) We say that a path [x —^r] (x e d V 2 f) is cold if it contains at least [0.4£] cold pairs (x, a). 

(v) Finally, we say that the pair {T,dv) e 5^^ x ^({-1, jg co/d if all the paths [x ^ r], with x e dV 2 /, 

cold. 


are 


The key result of this section is the following estimate. 

Proposition 3.4. We have 

p[(r,av) is cold] >i-r\ 

Then in Section l3^ we shall prove the following. 

Proposition 3.5. If{T,dv) e ^ 2 e ^ is cold then 

Let us see how this implies ProDosition l3.ll 

Proof of Provosition \3.1\ The first part of the proposition directly follows from the combination of ProDosition l3.4l 
and Proposition 13.51 For the second part of the proposition, we first need to introduce one more notation. For 
T e fPzt+i and j e[k- 1] we denote by T[j] e 32e the subtree of P pending below the j -th neighbor of the root. For 
a boundary conditon dv over dF, we denote hydv[j] its restriction to Tlj], 

We observe that if dv satisfies H, then so does dv[j]. Moreover, for any T e IJ 2 / and j e [k- 1], we have by 
definition of the Gabon-Watson process 


p[f[;] = r] <2P[r= F]. 

For F e Jat'+i and a boundary condition dv, using <3.31 and Taylor’s theorem, we get 


||^av_^(2.vl,|| 


, <8fcexp(2;6) sup 

;e[fc-l] J IJi 


Thereby, we obtain, using the previous observations 


> k^exp[2p)r^\ < [ 






<(A;-l)p[||v|l'j['-vg^j||oo>r 
< 2(A; - 1)P [ IIV®'''!' - v'2^’ Hoc > . 

For £ large enough, and using the first part of the proposition that we already proved, we have 

p [ II - v'2/> Hoc > (^ - 1)"^] < £-\ 

The second part of the proposition follows. 


□ 


Proof of Provosition \3.2\ For F e 52t'+2> let F' be the tree obtained by removing the tree pending below the last 
children of F’s root. Then if (F',dv") is good, then so is (F',dv'). In particular 

P [(F',dv') is cold ] > P [(F',dv) is cold ] 5 1 - £~^■ 


In this case we have 


l|v?,T-v'.3,^"«|| 


< £ 
1 — -c 


-1 


and moreover, applying Taylor’s theorem and by a similar token as previously, 

llpf.'*'' Hoc < A;exp(;6)F"^ 

This concludes the proof of the proposition. 


□ 
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3.3. Proof of Proposition l3.51 We begin with the following lemma, that shows that the messages exiting vertices 
X E Trunk! r, dv ) are under tight control. 

Lemma 3.6. Let[T,dv] x be fixed. ForallxeTiunk(T,dv], ivehave 

> l-exp(-k/3/2) and *(1) > 1 -exp(-fc/3/2). 


v^fci) 


The lemma will rely on a detailled analysis of terms of the form -. In order to simplify the discussion, we 

shall isolate this analysis in the following lemma. 


v;7(-i) 


Lemma 3.7. Letaclausea be fixed along with its adjacents variables x e da andafamily{Vx^a)x€da ^ ■ 

Let 

dgooda = {x^da,'Vx~aW > 1 -exp(-A:;6/2)}. 

LetiVa^xixcda ^ ^5*({-l, 1})*^ be defined by the following equations. 

^Sae{-l,l}*^ lsx=sV'a,/3(-^fl) nyeaa\{4) 'Vy^aiSy) 


^Sae{-l,l}*^ nye3a\{4 a(^y) 

Then for xe da, the following estimates hold true. 

(a) 

expi-p) < < exp(;6). 


(3.4) 


(b) Ifxed\a,then 

(c) If{x} = diaandd-iacdgoodCi,then 


'^a—‘x( 1) 

xCl) 


Vfl—x( 1) 

'Va^xW 


> 1 . 


'^a-’xi 1) 


> exp(0.99;S). 


(d) TfldiCa) \ {x} n dgood^l - P, then 




> exp(-exp(-pfcj8/3)). 


Proof. Point (a) easily follows from the fact that for any Sa e {-1,1}"^“, exp{-fi) < yfa.pifi < 1, and that if there is 
xeda such that Sx = 1, then y/a,pisa) = 1- 

Point (b) follows from the observation that if Sa,s'^ e satisfy Sj: = 1 and Sy = s'^ for y e da \ {x}, then 

V'a,pi.Sa)>yra,l}(s'fi. 

Point (c) follows from the observation that, if {x} = di a and d_i a c dgood'*. 

exp(-ls^i/3) < Y, isx=sVa,piSa) H Vy^al^y) ^ exp(-ls^p8) + 2fcexp(-fcj8/2). 

Sfle{-l,l}^« y^da\{x] 

Finally, point (d) is obtained by observing that, if |dia \ {x} n dgood'*! - P< 


Yi ^Sx=sVta,fii^a) "^y—a(^y) 1 

yeaa\{4 


- n (l-Vy^a(l)) < exp(-pA:;6/2). 

y£dia\{x] 


□ 


Proof of Lemm ddM We first prove the statement concerning . We prove it by induction over t - £ - 

For f = 0 the result holds by definition of Trunk(r,dv). Assume that the results hold for aU x e ¥21 such that 

dist(x, r) > 2{( - f ) and let x e ¥21 with dist(x, r) = 2(^ - r - 1) be fixed. We define 

Aix = djxn dix, 

Ai,ox = {a E djx, d_iflnTrunk(r,dv) = dja}, 

A_i,ox = {fl E djx, aEd_i_ox}, 

and for l< l< k, A_i /x - {a e d_i(x, a), |di(x)| = Z, |di(x) \Trunk(T’,dv)| < |di(x)|/4} 
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(3.5) 


We have, by Eq. 13.21 

,T,dv 




= n 


v2^(i) 


n 


A 


n n 


vlfd) 


n 




^,T,dVf 1-. ^,T,dv f T. ■*■■*• ■x',T,dv ( i-v -*•-*■ ^ ^,T,dv, i-. A J. ^T,dv, in 

f (-1) aeAi.oxV^ j (-1) aeAi a:\Ai,qa: j (-1) ;=1 aeA_i,;j: ^ (-1) aeA_i,oJ: ^ (-1) 


n 


vlfd) 

■~,T,dVf 


,CIV r _ 1 A 

aeaj;c\(Ai;i:ujL^A_i ,;c) ^a,\ '■ 

Because the messages (v, v) satisfy Eq. 0.41 . we can apply the result of Lemma l3.7l to them. It follows that 




> exp(;S[0.99|Ai,ox|-|A_i,ox|])exp 


k 

Y, |A_i,/A:|exp(-fc;S//4) 
U=i 


exp 


|-2/3|5jx\ (Aixu (uJLqA_i_ 


By definition of Trunk( T,dv), we have 


|Ai,ox| > L0.9A:J, |A_i,ox| < fO.lfcl, |djx\(Aixu (uJLoA_i,/x))| < 


Moreover, for 1 < / < fcwehave |A_i,;xl < |d_i_/x| < (byTR3) andhereby 

k k j^^+3 

Y exp(-A:;6//4) < Y —rp exp(-fc/lZ/4) < fc^exp(A:exp(-fc;6/4)) < 2. 

Replacing with these four estimate in 13.51 . we obtain -> exp{2kpi3), as desired. 

The second part of the lemma, regarding , follows from the observation that TrunkCT, dv) c TrunkCT, dv*°'). 


□ 


Proof of Provosition \3.5\ Let x e V 2 e \dV 2 e be fixed, and let for a e dj x and a boundary condition dv, : {-1,1} 

IR be defined by, for 5 e {-1,1} 


:iT,dv 


Saef-l,!)^" yeSja 


(3.6) 


By applying Taylor’s theorem to equation 13.21 . observing that (1) = 11 + Tla€d,x — 

X,l I I £ ■ (1) 




^a,\ 


we obtain 




II T,dv r,av™ II ^ 

II V ’, -V ||oo< sup 
ue|0,l] 


uv7’'(l) + (l-ii)v7''™(l) 


«v^f''(-l)+(l-u)vy7'“' (-1) 


r.avW), 


xA 


xA 




Y ®up 

ae5j;i: “£[041 


>T,dv -r,av™ 
S t — ^ ^ t 
CL, I Cl, I 


^T,dv , f, 

+d-M)e^,f 


We observe that for (xi,X 2 ,X 3 ,X 4 ) e [0,oo)^ we have 

(Xi + X2 + X3) 


< min{xi + X2 + X3, (xi + X2 + X3) < 1, 


(1 + Xi + X2 + X3)2 

uxi + (1-m)X 2 max{xi,X2} 

sup -<-. 

ue[0,l| UX 3 + (1- m)X 4 mm{X 3 ,X 4 } 

Using this in cunjunction with Lemma lT6l we obtain 

' “v^U-D+d-Uv^''™ (-1)' 

, “v7''(l)+(l-“)v7''™(l) ^ 


sup 

ue[0,l| 


1+ ■ 


uv‘^’^{-l)+[l-u)v 


r.avW) 

i,T 


(- 1 ) 


“v7''{i)+(i-“)vJ7™(i) 


<6exp(-fcjSl;ciscold/2). 


(3.7) 


(3.8) 






































We further observe that 


sup 

UE[0,1] 


-pT,dv _ -r,av™ 
^a,\ ^a,\ 




<exp(^) Iblf 


In particular, 

T A T’a,w( 0 ) 

* lloo£6exp(jS)exp(-fc;61;ciscold/2). 

Using again Taylor’s theorem, for any a e F 2 ( we have 

:4exp(-fcj8/2) Y, niin max|v^’^^(-l), (-1)} ||c 

yi^z 


lUr.dv --r.av™ I 
r^.t ^a.\ 


Therefore, if a is cold and the unique x such that dj a = {x} is not cold, we have 

T,dv _ .,r,av™ 

X,l 

xed^a xed^a 

Combining 13.81 with 13.101 . we obtain 


lUr.av _ -r,av™ I 

«,t a,\ 


<exp{-kp/2) Y l:cnotcoldl|v;7’ 


I V' 1 lu.Tdv I 

+ Y Ijc is cold 11^^ j ^x,] 


(3.9) 


(3.10) 


'^x',\ I oo ~ ^ ^ is cold + Ijcisnot coldlfliscoldly isnot cold)(2) Ijv^’j V^’j |j^. 

ae3j;cye5ja 

Iterating this equation, we obtain, (using that for any x e dl^^. the path from the root r of T to x contains at least 
L0.4^J cold pairs (x, a)) 

||vf.yj-v'2f)|| <24^exp(;S^) Y exp(-A;;SL0.4^J/2)||dV;,-dvf 11^ 

’ “ x€dV 2 e 

< |dV2t'|24^exp(^/)exp [-kp[0A£\ 12] 

< (dfc)^24^exp(-O.Olfc^^) = oy(l). 


□ 


3.4. Proof of Proposition l3.41 

Proof. In order to prove the proposition, we will need to slightly extend the notion of cold variables and cold 
clauses. Let a pair {T,dv] e 32 e x ^5®({-l, 1})'^^ be fixed. Given v e T and {w] - dj v, let Tp denote the subtree of 
T\{w} rooted at v. Also recall that we denoted by dV 2 / = dT. 

For {a, x) with {x} = dj a, we say that x is strongly cold with respect to a for the pair (T, dv) if there exists no tree 
T' E 32 ( and no boundary condition dv' over dV 2 ^’ such that the following is true. 

• X is not cold in (T'jdv'), 

• Tx = Tx, 

. yxedTx\dTa,{dv')x^(dv)x. 

Observe that strongly cold variables are also cold. Let, for d e djx, p^.a! be the probability that x is not strongly 
cold with respect to d when the pair (T, dv) is drawn at random. We shall prove by induction over f e {1,..., ^} that 
for X at distance 2t from dV 2 £ and a' e d i x, p^.a' - 2“° ®*. For t-0, the result follows from the assumption on the 
distribution of dv. Let us now assume that we have proved the result up to f > 0 and consider x at distance 2( f + 1) 
from d V 2 ^’ and a' e dj x. For x not to be strongly cold with respect to d, one of the following must happen. Let 
be the set of strongly cold variables at distance t from dV 2 ^- Let a be the clause such that dj x = {a}. 

(a) there are less than L0.95A;J clauses a e dix such that dia = {x}, 

(b) there are more than fO.OSfc] clauses aedx such that |d_ial = k, 

(c) there is 1 < / < fc such that there are more than 0.5k^'^^ll\ clauses a e d_ix with |dial = I, 

(d) |{hEdi,ox\{a},dh\{x}5Z V'h"h}| > 

(e) |{hEd_ix, |d_ih|<fc, |dih\{x}\y'^'-i>|>|dih|/4}|>fc3/4-i. 

12 









By definition of our random process, (a), (b) and (c) each hold with probability at most 2 The probability 

that a given clause b e difiX \ {a] contains at least one not strongly cold variable different from x is Zyeaib Py,b + 
By definition, the probability that each of the clauses hi,...,by e diox\ {a] contain at least 
one not strongly cold variable different from x is upperbounded by Therefore, the probability 

that (d) happens is at most Similarly, (e) happen with probability at most Therefore we obtain 

Px,a' - 3.2“° ®^*^ + 2.2“^-®^ < 2“° ®^, as needed. 

Let a clause a e F 2 ( be fixed as well as x e dj a. Let d\a- {y}. We say that a is strongly cold with respect to x if 
|dia \ {x,y} n ^ 1- Let qa,x denote the probability that a is not strongly cold with respect to x e dja when the 
pair (r,dv) is drawn from a distribution that satisfies the hypothesis of the proposition. By our previous estimate 
(g) and (h) happen with probability at most while (f) happens with probability at most 2“° ®^. In particular, 
qa,x ^ By construction of the strongly cold clauses, the probability that a pair (x, a) with [a] = dj x is not 

strongly cold with respect to h e d i x is then upperbounded by Px,b^a,x ^ 2 ~^''^^. 

Let X E d V 2 / be fixed. Recall that we denoted by r the root of T. We denote sequence of variables and clauses on 
the path [x ^ r] by (xq = x, ao, xi, ai,..., r). For the path [x ^ r] not to be cold, there must be at least pairs 

(Xj, a,) along this path that are not strongly cold. Moreover, for i\^ ^ ii, the probability that the {Xi.,ai.) 

are strongly cold is independent (by construction). Therefore we obtain 

P [[x ^ r] not cold] < ^ ^ P [(x^, not strongly cold and (x,^, fljj) not strongly cold and 

/>L0.6fJ 0</l< —<!;</ 

... and (Xi,,) not strongly cold] 

' r 1 

^ E E n P I {Xi -, Ui .) not strongly cold I 

;>L0.6fJ 0</i< —<!;</j = l 
< 2^ < 2“^-°^^^ 

Consequently, we obtain with the union bound 

P [(r,dv) is not cold] < ^ P []x ^ r] not cold] < [dVaf = o/(l). 

x€dV2e 


□ 


4. The fixed point problem on trees 


In this section we prove Propositio n 1.2\ and Lemm d2^ 

We shall obtain the propositions by making the connection between the skewed fixed points of the operator 
'^k.dF ^nd the analysis of Belief Propagation on random Galton-Watson trees studied in the previous section. We 
first identify 1}) with (0,1) through q ^ pf-l). This also identifies 1})) with &*{0, 1). With the 

notations of the previous section (and, in particular, q given by 13.11 1. we shall prove that 


Proposition 4.1. Letn be a skewed fixed point of^k^dF tind£> 1 be fixed. Then we have 


”= E p'S/oj 


S dv 


X€dV 2 f 


l-dVx{-l) dVj|;(-l) 

;-:- d7t{dvx) + lbr_,=i - dnidvx) 


l-q 




q 


Let us see how Proposition ! 1.2l follows from this proposition and from the result of the previous section. 


Proof of Proposition ] 1.2\ Let, for £ >0, e ^3*({-l, 1}) be the distribution of Let v be distributed according 
to 71. It follows from Proposition 14.1! that v = vp', where T and dv satisfies the assumptions of Section [3] In 
particular it follows from Proposition 13.1! that 71 weakly converges towards hence the unicity of the fixed 
point. By a similar reasonning, we see that 71 ^^^^ admits a weak limit, proving the existence of the fixed point. □ 


4.1. The multi-type Galton-Watson branching process: proof of Pronosition lT. 1 1 For 7 t, 7 ie £^[0, 1) we define 


I 

j(0, 


hiji) = / T]d7i(7]'), hljr) = 


f 

Ji0,l 


7]d7i[7]]. 
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We let /; (0,1)^ ^ ^ (0,1) (resp. /: (0,1)*^ ^ ^ (0,1)) be defined by 

^ i-cpUtIVi 

z{r]i,...,r]d-i) z[T]i,...,r]k-i) 

and fdtfk.p '■ [ 0 , 1 ] ^ ( 0 , 1 ) be defined by 

fd{ri) = = fk,ii ( 77 ) = /( 77 ,..., 77 ). 

We say that {n,n) is a fixed point of {^k,d,^t^k,d,^) iff ^ = ^k,d,p&) n - ^k.d.pi^)- 
Lemma 4.2. If{n,Ti) is a fixed point of i^k,d,p<'^k,d,i 3 )> thenwe have 

h[7i] = fd(h[ji]), h[Ti] = fk,p{h[7i]). 

Proof. We first observe that, using the multilinearity of z (resp. z) 


d-l 


zm 


= I Z(77i,...,77d_i)(^d7f(77/) = Z(fi[7T],...fi[7r]), 

j=i 


p k— 1 

Z[7r] = I Z(77i,...,77fc_i)(g)d7r(77/) = Z(/2[7t],...,/7[7t]). 

d(0,i)*-i 7=1 

Using these equations, we obtain 

h[7t]= [ pd.^d,k,/3l^Kr]) = f z[fji,...,fjd-i)f{f]i,...,f]d-i)(^dTt{f]j) 

Z[7i] ^1 

Z[7r] ^( 0 , 1 )“^-' 

= fd(h[7i]). 


rf/ 2-1 d-l 

n hj n 

y=l J=dl2 


d-l 
10 
7=1 

d-l 

(^dnifjj) 

7=1 


Similarly, we have 


I 

J(0,1 


hl7t]= I pd^d.k. 

M) 


1 f 

p[7i]{Tj) = I z(77i,...,77fc_i)/(77i,...,77j;_i)(g)d;r(777) 

Z[7r] J(o,i)*^-i /Ti 

-f 

[tt] d( 0 ,l)*:-l 


ZlTT], 

■ AiBWtt]). 


k-1 

i-c/jD'u 

7=1 


k-1 

0 d7r(777) 

7=1 


□ 


Recalling the definition of <7 = q[d, k, f) in Eq. 13.11 . and defining q- I- q, the following is a simple observation. 
Fact 4.3. The set of equations 

y = i-y. y^fk.piy), 

admits for unique solution in [ 0 , 1 ]^ the pair (q, q). 

We define the measures 7 i+, 7 i-,ji+ and Ji- over (0,1) by 


1 - 77 77 

d7r+(77) = --d7r(77), d7r_(77) = -d7r(77), 

l-q q 

I -77 rj 

d7t+ ( 77 ) = -—:: dTT ( 77 ), d7r_ ( 77 ) = - dTT ( 77 ). 

1-77 77 

Lemma 4.4. If{n,n) is a fixed point of {.^k,d,p>l^k,d,i})> tve have 

d/ 2-1 d-l 


n aiz—L a 

.0 d5f_(777) (g) d^+(777), 

d{ 0 ,l)'^-l 7=1 j=dl 2 


(4.1) 

(4.2) 

(4.3) 
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(4.4) 


n. 




r=\ 


d/2-1 d-1 

d^+(%) <S> d7T_(77;), 

j=l J=dl2 

k-l-r 


r J 1 -cpq 


k-l 


I 

-f(0, 


iJ j=l j=r+l 


(4.5) 


+ exp[-p) 

= L 


^k-l 


l-cpq’' 




r=0 


k-l-r 


L 


q^a-q) 


r k—1 

.,.-1 . v.-o 

tu.i) ]=1 ]=r+l 


(4.6) 


Proof. We first observe that, recalling the definition of z in Section fL^ qZlri] - q‘^^^ '^(1-We then compute 
7r_=/' -SrjdTTiq]^ f -Sr,d^ic,d,6l^]('n'> 

•'(0,1) ^ J(o,i) q 


.. rj,^lSdniqj) 


r nffr'^infr^/ 2 (i-%) 4-1 

' i(o D'i-i fi a) 

•^(0,1)'' 1 iL 1 1 = 1 


] 

rf / 2-1 rf -1 

^nm.-,^d-i) <S> dt_(f7;) 0 d7T+(f7j) 

j=l j=d/2 


= f 

J( 0 ,l)‘l -1 

The equation on n.^. is proved similarly. We also compute 

71 -=/' ^( 5 ^d 7 T( 77 ) = f ^Sfid^icd.pMiV) 

•'(0,1) f •/(o,i) q 


r l-Cfilll?; 

. 


I k - q’’ [1 - q) 


k-l-r 


r=o\ 1" j ^-Cpq 


k-1 


L .Tl.-l) 

•'ll),'; j=l j=r+l 


+ exp(-/3] 


. 

The equation on is proved in a similar manner. 

Proof of Provosition \4.1\ We observe that tt = (1 - q)n+ + qji-. Replacing with Lemma fOl yields 

ir= E Pkds^'^^f 0 [lfc;ci=-i ^ ^^ d7r(dV;c) + l&^,=i ^^"'^ ^^ d 


□ 


d7r(dv;c) 


- jctyv '2 ' • 

By induction over 1 < r < ^, using repeatedly Lemma l4~4l we obtain that 

^ f, l-dvA-l) 


11= E f " flv w 

reS'at ’ •/^({-l,ll)‘’''2t T 


l-q 


dVj:(-l) ) 

d7i[dvx] + lb;,,,=i-d7r(dv;c)j 


This concludes the proof of the proposition. 


□ 
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We define, for (vi,..., v^, vi,..., e ^^({-1, 1 })^+'^ and {bi,..., b^) e (-1,1}*^ 


zi(vi,...,vd)= n n n n -vji-D, 

7 <d/2 j>dl2 j<d!2 j>dl2 

k 

Z2{vi,...,Vk,bi,...,bk] = n 

j=i 

•Z3 (Vl, Vi) = Vi (1) Vi (1) + Vi (- 1) Vi (- 1). 

In order to prove Lemmawe also recall the following standard result, which we prove in Section!?] 

Proposition 4.5. We have 

Proof of Lemm d2^ We compute 

InE [zi (Vl,..., Vd)] = In ^ j _ 

lnE[z 2 (vi,...,Vfc,hi,...,hfc)] = ln(l-c^^ 7 ^|, 


lnE[z 3 (vi, Vl)] = ln(2^(l - q)). 


Thereby we have 




The proposition then follows from Prooosition l4.5l 


□ 


4.2. Finite £ approximations of SS[k,d,P). We finally present a simple approximation of SS{k,d,l5) that will he 
useful in the following. Recall that GW{k,d,p,2£) and GW'{k,d,p,2£) were defined in the previous section. Let 
(Ti,... Trf) denote the probability that the neighborhood of the root in is equal to (Ti,..., T^) under the pro¬ 
cess GW'(k,d,p,2£ + 2). Denoting by ei the first edge exiting the root of the random process GW (k,d, p,2£ + 2), 
let for T £ ^ 2 / and T e Sze+i p[T, T] be the probability that the 2^-neighborhood (resp. 2£ -i-1-neighborhood) 
of this edge when removing its clause node (resp. variable node) is formed of the tree T (resp. T). Similarly, de¬ 
noting by fli the first clause connected to the root of the random process GW'{k,d,p,2£ + 2), let for T\,...,Tk e 
3'( Pk,d,p(Tiy - Tk) be the probability that the 2^-neighborhood of ai is equal to (Ti,..., T^) under the process 
GW{k,d,p,2£ + 2). 

Finally, let ^ be the unique skewed fixed point of “^k.d.p and, for the ease of notations, let Ti+,n-,n+,n- 
denote the quantities associated to ^ ^ through Eq. I4.1H4.2L 

Lemma 4.6. We have 


Sg{k,d,p)^ Y, ^ pf^d^phfi,...,fd) [ ^ln[zi(v|''\...,v|''‘')] (g) dn+[{dvj)x) (g) dn-[{dvj)x) 

(fl,...,fd)ejr2^^j ’ ’ ^ i = ix€dlfj X€d-lfj 




(^ dn+{{dvj)x) dn-i{dvj)x) 

j=lx€diTj xed-iTj 


+ T T. Pfdp^T,,...,T,)[ ln[z2(v^^;S..., 

^ r r 1 ^ 

Y_ 2^n[z3(v^^\v|''^)j (g) d7i+{{dVj]x) (g) d:?r_((dv^);c)-t o/(l). 

re5"2^,feS‘2(-+i ’ ’ "t(0,lp j=lxedifj xed-ifj 

Proof. The proof is obtained by writing the expectation values in the definition of SS{k, d, p] explicitly in terms of 
7i+,7i-,7i+,7i- and by following steps similar to the one of the proof of Proposition l4.ll □ 
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We define 






(2e+\) ^(z^+Dd 

fi ’■■■’ fd ’\ 


(ri,...,rrf)e3-2“^j 

+ I E oSVn. nnnlMy-^p .v?«,| 

reS^^,feS^^+i 


Proposition 4.7. Wfe iiafe 

3S[k,d,p) = (k,d,p] + o^(l). 

Proof. The result easily follows from the weak convergence of toward ^ □ 

We now proceed to prove Prooosition l2.8l In order to do so, we need to state here a standard lemma about the 
local convergence of the random formula <1>, that we will prove in Section[6](see Lemma lO] . 

Lemma 4.8. For all £ > 0 and\/T e ^ 21 + 2 , we have pctiiT] ~ 

Proof of Provosition \2 . 81 By the previous lemma we have, for £>Q, 

lim —W.[SS^p]-f^^^\k,d,B). 

n—oo fi 

The result then follows from Prooosition l4.7l □ 


5. Marginal ANALYSIS 

We wiU exhibit a number of deterministic conditions (six in total) that entail the non-reconstruction property. 
Subsequently we are going to show that the random formulas ® and ® enjoy these properties with high probability. 

Then we will need some information on the local structure of a formula ®. For a variable node xeV and f > 0 
we let denote the t-neighborhood of x in ®. For a tree T e 32(+2 (defined in Section l4l2i we define the 

empirical density of T by 

1 ^ 

P<t(T)-— 2^ lA(2'’+2)<p(;Ci)sr- 
^ i€ln] 

We shall say that a regular fc-SAT formula ® satisfies property Local Structure if the following is true, for every £ 
large enough. 

Local Structure: VT e 52^+2, p<tiT) ~ 

We shall also demand that ® satisfies the Cycles property: 

Cycles: There are o[\/n) cycles of length at most \/\nn. 

In order to proceed further, we need to introduce a few more notations, similar to the ones that we used in 
Section|3] Let ® be fixed and V denote its set of vertices, F denote its set of edges and E denote its set of (undirected) 
edges. For x e 1^ (resp. aeF), we let 

d\x- {aEdx,ha,x- -1], d-\x- {aEdx,ha,x-Pi, d/x = {a e dx,|{y eda\ {x},= 1 }| =/}, 

di a = {x E da, ba,x = -1}, d_i a = {x e da, ba,x - 1}. 

We also introduce di_/x = dixn d/x, d_i ;x = d_ix n d/x, and for ao e dx fixed, di(x, af\ - d\x \ {aol, d_i(x, ao) = 
d_ix\ {aol, di,/(x,ao) = di,/x\ {aol, di,/(x,ao) = di,/x\ {ao}. 

We define the X-coreoi ® (in symbols: Core;^!®)) as the largest set W of variables such that all x e W satisfy the 
following conditions. 

i-l 

CRl: there are at least A:(l - ^) clauses a e dix such that dia = {x}. 

CR2: there are no more than kexpi-f) (l + clauses a e dx such that |d_i a| = fc. 

CR3: for any 1< l< k the number of a e d_ix such that |dial = Z is bounded hy Xk^^^! l\ . 

CR4: there are no more than clauses a e dix such that |dial = 1 but da gt W. 

CR5: there are no more than clauses a e d-ix such that |d-i a| < fc and |di a \ W| > |di a\ /4. 


17 


The A-core is well-defined; for if W, W' satisfy the above conditions, then so does W u W. Further, if A < A', then 
Core;i (<F) c Core;i' (<5). Also note the similarity with the trunk of trees defined in Section l3^ We say that <1) satisfies 
the property Core if and only if 
Core: |Corei/ 2 ('l')| > (1 - 

Our aim wiU be to identify a large set Vgood c 1/ of vertices whose value under a typical assignment in the cluster 
is unlikely to be very far from the planted one. A first candidate for vertices whose marginal may go wrong are 
those which do not belong to the 1-core of <!'. Yet, we are not guaranteed that vertices in the core have marginals 
sufficiently close to . For instance, if the marginals of most of the neighbors of a given vertex x e Corei I®) went 
astray, there would be no reason for x’s marginal not to go astray itself. However, we see that the vertices in the core 
whose marginals are not what we think they should be must clump together. We say that a set S c H is X-sticky if 
and only if for all xeS, one of the following conditions holds true. 

STl: there are at least Afc^^^ clauses aedix such that dia- {x] and d-ian S ^ 0. 

ST2: there are at least Afc^^^ clauses a e d_ix such that \d-ia\ < k and Idian S| > \d\a\IA. 

We say that ® satisfies the property Sticky if and only if 

Sticky: ® has no 1/2 sticky set of size between and 

Finally, say that a variable xeV is {e, 2£)-cold if the following is true. Let T - A^^x. Then T is a tree. Moreover, 
if we choose a boundary condition t such that 

• the values of variables y that do not belong to the core are chosen adversarially, 

• the values of the other variables are chosen i.i.d. such that the probability of -1 equals exp(-2;6), 

• subsequently an adversary is allowd to change some of the - Is to -i- Is, 

then with this boundary condition the BP marginal at the root of the tree is within e of p j- in total variation distance. 
We say that ® satisfies the property (e, 2f') -Cold iff 
(e, 2^)-Cold: All but en variables are (e, 2^)-cold. 

We say that a formula ® is (e,^)-tome iff the properties Local Structure, Cycles, Core, Sticky and (e, 2^)-Cold 
hold. Planted formulas are likely to be tame. 

Propositions.!. ForanyoO, there is i > 0 such that w.h.p. <S> is (E,£)-tame. 

Similarly, formulas from the planted replica model are likely to be tame as well. 

Proposition 5.2. ForanyoO, there is £> 0 such that w.h.p. Q> is[e,()-tame. 

We prove Propositions 15.Il and l5.2l in Section^ We are going to show that (£,^)-tame formulas have the non¬ 
reconstruction property. In the rest of this section, we assume that ® is (£, -tame. Let us briefly write | • ] = [ • 1 
and <■> = (Aip./s- 

We say that a set T c Corei (®) \ Si (®) is a-closed if for any xeT and all aedx we have 

{y e dan Corel!®) \ S(®): (7(y) = -1} c T. 

Moreover, for a clause b we say T c Corei (®) \ S(®) is (cr, b) -closed if the above holds for all x e T and all aedxib. 

Lemma 5.3. Suppose thatch is (e, £) -tame. Then for any a such that 1 • <7 > (1 - n and for any (a, b] -closed set 
T c Corel!®) \ S!®) the following is true. Letaix) - !-1)^*^^^*(t!x). Then 

E^[d)<E^[(j)-k^''^\T\. !5.1) 

Proof. Consider the following process: 

• Let ao - O', Vo - T and Uo - cr~^ !-1) \ Vq. 

• While there is it e Vt such that i:(i)!!-l)^*'^*'*(7t!-)) ^ F’tplo’f) - pick one such it uniformly at random 
andleto-f+i!-) = (-l)^’'"*'’o-t!-) and Vf+i = VtMit). 

Clearly, 

JScplcrf) < JScplcr) - !5.2) 

Let T be the stopping time of this process and assume that x < | r|, or, in other words, that P,- ^ 0. We claim that 
is a 1-sticky set. Indeed, because T is cr-closed for i e we have 

< £'(i,!!-l)^‘'=''(Tf !■)) - F’fl.lo'r) < l{b E d/}-|di,o!/)| -I- \{a e di,o/,d_ian IW- u Ho) ^ 011 
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+ |a_i,o/| + I Ui</<fc {a e d'_^ i,dia<z Vr u Uo)]\. 

Because i e CoreiC®) we have |5i,o«l 5 |5_i,o«l < 3, |{a e di Q,d-ian Uo^0}\< and \{a e di oi, |d_ian Uo \ > 

|d_ia|/4}| < k^^^. Therefore, one of the following must hold. 

(a) \{a£difi,d-iar\Vr^ 0 ]\>k^'^, 

(b) \{aEd\fli,\d-iar\ Vr\ > |d_ia|/4}| > 

It follows that the set Vr is 1-sticky. However, Corei (<1>) \ Si ($) cannot contain a 1-sticky set of size \ Vr\<\T\< 
as this would contradict the maximality of S(<1>). It follows that t = \T\, and therefore cTj = cr, from which I l5.ll 
follows using 15.21 . □ 


Fact 5.4. For any variable x the following is true. Let jix, L) be the number of trees of order L rooted at x that are 
contained in the factor graph o/®. ThenY(x,L) < L(lOOdfc)^. 

Write T (x, cr) for the smallest cr-closed set that contains x. In other words, this is the - 1-component in Corei (®) \ 
Si (®) that X belongs to. If cr (x) = 1 we let Tfx, cr) = 0. 


Lemma 5.5. /f® is (e, (]-tame, then for all x e Corei (®) \ Si (®) we have 

[cr(x)] > 1 -exp(-/iA:^^'*/2) and [l{|r(x,£r)| >Inlnn}] < l/lnn. 


Proof Let N - 2~^^^n. Because ® is tame we have |1 -cr < n- AT] < exp(-Q(ri)). Therefore, |l{|r(x,£r)| > N]j < 
exp(-Q(n)). Hence, let t < AT and let 6 be a tree of order t with root x that is contained in the factor graph of 
® and whose vertices lie in Corei (®) \ Si(®). If cr is such that T{x,a) = 6 , then Lemma |5j3] implies that (t(x) = 
^_l'^i{x€T{x,a)} S 2 |-jsfjgs £([,(ct) < ilcpCcT) - Consequently, 


(l{<r = cr}) 


< exp(-/lfc^^^r). 


Hence, by Fact l5.4l the union bound and our assumptions on f and d, 


{l{\T{x,a)\^t}) 


< t(100dfc)^exp(-ySA:^'^f) <exp(-0.99;Sfc''''‘t). 


,3/4, 


<1{<7(X) = 1}> 

This bound readily implies the second assertion. To obtain the first assertion, we sum 15.31 over l< t< N. 


(5.3) 

□ 


Fact 5.6. Let q e (0,1) and L > 1. Suppose that p is a probability distribution on {+1}^ such that for any i e [L] and 
any yi,..., yi e {+ 1 } we have 


(l-£ 7 )p(yi,...,y;_i,-l,yi+i,...,yi.) < qp(yi,...,yi_i,l,yi+i,...,yi,). 
Furthermore, letv be the distribution on {+1}^ such that for all yi,... ,yi, e {+1} we have 


(l-q)v(yi,...,y;_i,-l,yi+i,...,yL) = qv(yi,...,yi_i,l,yi+i,...,yL). 
Moreover, letB c {+1}^ be a set such that for all beB and all b' > b we have b' e B. Then p[B) > v{B). 


Lemma 5.7. Let r be a variable for which the following conditions hold. 

(1) r is {£,f]-cold. 

( 2 ) r has distance at leastln^^^ n from any cycle of length at most s/\nn. 

LetTr be the event that a is a good boundary condition for r. Then (\{a 0Y r)) ^ 2c. 


Proof. Let X be the set of boundary variables. Moreover, let si be the event that max^^ex ir(x,(T)| < Inlnn and that 
a-\ > (l-2“^^^)n. Because ^ is bounded, Lemma lsTsl and the union bound imply that f llo' e siW ~ 1. Furthermore, 
if sd occurs, then our assumption ensures that the subgraph of the factor graph induced on F = r u Uxex T (x, tr) 
is acyclic. 

Now, fix a variable x e X and a esd such that cr(x) = -1. Let a be the clause that is adjacent to x on its shortest 
path to r and let T[x,a,a) be the smallest (cr, a)-closed set that contains x. Further, define (f(y) = 

Then Lemma |5(3] shows that ficplCT) < r(x, a,cr)|. Moreover, because the subgraph induced on Y is acyclic we 

have d{x') - cr(x') for all x' e X \ {x}. Consequently, by Fact l5.4l and the union bound. 


(1{<7(X) = -l}nyex\{4 l(o-(y) = (t{y]}l{a e si]) 
(1{<T(X) = l}nyeX\{4l{o-W = 


< Y. f(100dA:)^exp(-;6fc^'^t)]<exp(-;Sfc^'^/2). 

t<lnln« 


(5.4) 
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Since [1{<7 e si}\ ~ 1 and because for all t : X ^ {+1} we have 

/ Y[ l{o-(y) = TM}\>exp(-dfc)S|X|)=0(l), 

\yeX\{4 / 

15.41 implies that for any t, 

(l{<T(x) = -l}nyeX\{4l{0'W=T(y)}> 3/4,0, 

—- - -— < exp(-/3fc^'^/3). 

(I{t7(x) = l}nyEX\w l{o'(y) = T(y)}) 

Thus, the assertion follows from Fact l5.6l □ 

Finally, Pronositions l2.5l and l2.6l follow from ProDositions l5.1l and l5.2l and Lemma lSTTl 


6. Typical properties oe the random formula 

In this section we prove Propositio ns. l\ and Provosition \5.2\ . Let Sn.k.d denote the set of regular fc-SAT formulas. 
For V eVu P and £ >0, we let v (resp. v) denote the set of vertices at distance exactly £ (resp. less than £) 
from V. 


6.1. Proof of Proposition IS. 1 l and Proposition l5.21 We first deal with the easiest condition Local Structure. 
Lemma 6.1. W.h.p. ® Local Structure. 

Proof. Let xeV and T e ^ 2 {e+i) be fixed. Let XxiT) be the number of formulas ® such that = T. It is 

straightforward to compute that there are precisely 

~2(f+i).,j,,_ {ndUy? _ 

Pk.dp ^^\ndl2-e+)\{ndl2-e-)\^^ ” 

ways to construct a tree of depth 2{£ + 1) around x, where e+ (resp. e_) is the number of positive (resp. negative) 
literals that appear mT\dT. Once this as been done, it remains to connect the {dnl2-e+] positive litterals clones 
(resp. [dnl2- £-) negative litterals clones) together. This yield 

XxiT] [ndl2V? {ndl2-e^)\{ndl2-e.)\ nil = 

\Sn,k,d\ {ndl2-e+)\{ndl2-e-)\ [ndl 2 )d ^ Pk.d.r^^ 

Consequently, we have 

Moreover by standard concentration arguments p® (T) is concentrated around its mean and we have w.h.p. 

P®tr)~pf,y(r). 


This holds for any T in the finite set (? 2 (/+i) > ending the proof of the lemma. □ 

In particular, this entails the following. 

Corollary 6.2. W.h.p. ® satisfies Local Structure. 

The following is a standard result. 

Fact 6.3. W.h.p. and ® satisfy the property Cycles. 

We will prove the following in Section lO] 

Proposition 6.4. W.h.p. ® and ® satisfy Core and Sticky. 

The remaining of this section is devoted to a proof of the two following lemmas. 

Lemma 6.5. For all e > 0, there is£ >0 such that w.h.p. ® is (e, £)-cold. 

Lemma 6.6. For all e > 0, there is£ >0 such that w.h.p. ® is (e, £)-cold 

Proof of Provosition \5. h and Provosition \5.2\ The propositions immediatly follow from the above lemmas. □ 
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Let a > 0 and (zi,...,Za) e be fixed. Let A - {y e V,3l £ [a\,y £ and for y e A let be the event 

that y e Corei (®) \ Si (®). Moreover, let ® be the event that zi, Z 2 ,..., Za are at distance strictly greater than one 
from the other in and that their neighborhoods are tree-like. For y e A, let also denote the cr-algebra 
generated by the function (®, zi,..., Za) ^ (A^^^'<l>(zi) U • ■ • U A*^^’$(Za)). 

Lemma 6.7. For y£ A, we have 

Proof. Let ay £ dy be such that ay £ U;e|a] A^^^'®(z;). Let O' be obtained from O by the following operations. 

• Select jc E O and ax e dx uniformly at random. 

• Replace the pair of edges {(y, ay), (x, ax)] by the pair of edges {(y, ax), (x, ay)}. 

Let S be the event that O' satisfies Si. We observe that P[@] = 1 - o„(l) and P[^] = 1 - o„(l). Conditioned on Si,S 
and J^y, O and O' are identically distributed. Moreover, we have 

Corei /2 (6) \ Si /2 (6) c Corel (O') \i S(O'). 

It follows that 

P [^‘€y\S>,3^y] = P "t 0„ (1) 

< P [xe Corei/2(0) \Si/2(0)|@,<g’,.^y] -i- o„(l) 

< P [ X e Corei /2 (O) \ S 1/2 (O)1 3^y\ + On (1). (6.1) 

For a fixed cr-algebra ^ generated by (O,zi,..., Za) ^ (A^^^'O(zi) u • • • U A^^^'O(Za)), let denote the event that 
there is O" isomorphic to O such that ^y = Then, because x is a random element of V, we have 

P [ X C Corei /2 (6) \ S 1/2 (6) | .^y = = P [ x C Corei /2 (6) \ S 1/2 (o) | je\ 

= P [x C Corei /2 (O) \ S 1/2 (O)] -t o„(l), (6.2) 

where the last line used that P {J€\ = 1 - o„(l). Finally, Proposition l6.4l implies that 

P [x e Corei /2 (6) \ Si /2 (O)] < + o„(l). (6.3) 

Combining <6.11 . 16.21 and 16.31 concludes the proof of the lemma. □ 

Proof of Lemm dKR For £ > 0 fixed, let F = [{x e V, A^^^^O(x) is not (£,2^)-cold in 0}|, and let a(n) be a slowly 
diverging function. We are going to show that there is a sequence y/ = o/(l) such that 

E[F(F-l)...(F-a:-H)] < (y/n)". (6.4) 

This bound implies the assertion; indeed, 

P[F>3y/n] <P[F(F-l)...(F-a-H) > (2y/n)“] 

E[r(r-i)...(r-a + i)] 

■ (2y^n)“ 

To prove <6.41 . we observe that F(F - 1)... (F - a - 1 -1) is just the number of orderer a-tuples of variables such that 
A(2fl(|)(x) is not (£,2^)-cold. Hence, by symmetry and linearity of expectation, 

E [F(F - 1)... (F - o: -t 1)] < n“P [Ti,..., are not (£,2^)-cold], 

where T\,...,Ta are the 2£- neighborhoods chosen of a random vertices xi,...,Xa of F. Let S be the event that 
xi,...,Xa are at distance greater than 5£ from each others and have tree-like ftd neighborhoods, and let A = d Ti u 
■■■udTa. Then Lemma lfiTTl imnlies that for j £ A 

In particular, using Lemma leTI we can apply the result of ProDosition l3.2l to obtain that, for 1 < i < a, 

P [Ti is not (£,2^)-cold|®, Ti,..., T;-!, Ti+i, r„] < r\ (6.5) 

We have 

P[Ti,...,Ta are not (£,2^)-cold] < P [Ti is not (£,2^)-cold]®] P [72 is not (£,2/)-cold|ri is not (£,2^)-cold(, @]... 

P [Ta is not (£,2^)-cold|ri,..., Ta-i are not (£,2^)-cold. Si]. 
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Using I l6.5t this yields 


¥>lTi,...,Ta are not {e,2^)-co\A\m < (o/(l))“. 

Along with the observation that P [@] = 1 - o„ (1), this concludes the proof of the proposition. □ 

In order to prove Lemma lhhl we need to extend Lemma lSTTl to the planted replica model. This will require a few 
more auxiliary results. We say that a tree T e is 2£-pure if and only if 

> l-exp(-100/1). 

Let ^ 2 ^ ^ 2 e denote the set of pure trees. Let, as before, a > 0 and (zi,..., Za) e V" be fixed, as well as a formula 

and an assignment (<l>,cr) e x {-1,1}”. Let A = {y eV,3l e[a],y e andfor y e A, let Uy be the unique 

clause in dy n A^^^'<l>(z/) and let ly e [A:] (resp. I’y e [d]) be such that y appears in ly-th position in Uy (resp. fly 
appears in ly position in y). For y e A, let A*^^^®(y ^ fly) denote the 2£ neighborhoof of y in the formula where 
the edge between y and fly has been removed and let 

• £!^y be the event that A*^^^ ® (y ^ fly) is tree-like and is 2f'-pure in ®, 

• SSy be the event that oi(y) = 1, 

• '^y be the event that y e Corei/ 2 (®) \ Si/ 2 (®). 

Moreover, let @ be the event that zi,...,Za are at distance greater than 5£ in ®. For y e A, let also '^y denote the 
sigma algebra induced by the functions 

(®, Zi,..., Za,y) -- (x = (Zi) U ■ • ■ U A'^^^®(Za) \ A^^^’oCy ^ fly),cr|x) ■ 

With these notations in mind, we will prove the following. 

Lemma 6.8. For ye A, we have 

Proof. This lemma follows from Lemma lS^ and Lemma |3^ using in addition the fact that P [Si] = 1 - o„ (1). □ 

Lemma 6.9. For ye A, we have 

P[^SSy\s^y,S,^y]<A~'‘. 

Proof. Let ^ 2 /+! denote the set of 2f' -i-1 neighborhoods of the first children of the root of the Galton-Watson 
process GW{k,d,f,A£) (defined in Section]^. For a probability distribution p over {-1,1}^ and 1 < / < fc, let p[l] 
denote the Z-th marginal of p. Finally, recall that for ae F, Pa^^^^ was defined in Section l23] 

Recalling the definition of the replica planted model, we have 

¥[-^9Sy\sdy,S,^y]^ ^ P [ ® (fly) = f|.«Zy, 'i^y ] [Zy] (-1) . 

For T e Z52f+i and 1 < Z < k, let r[Z] denote the subtree of size 2£ rooted at the Z-th variable node adjacent to the 
root of T. For T e let 52^+i (T, Z) c ^ 2^+1 denote the set of trees compatible with T on Z-th position: 

5^^+i(r,Z) = {fEf2^+i,f[Z] = r}. 

Then we immediately deduce from the previous equation that 

P[-'%|.!4,.^y] < sup ^sup /Ip^’^^^[Z](-l)(l-to„(l)) 

Using the definition of A’-pure trees, and the observation that marginals and messages cannot differ by a factor of 
more than exp(jS), for any T e ST.^^ and T elJ}[T, Z) we have [Z](-l) < exp(-5Oj0) < 4“*^, ending the proof of 

the lemma. □ 

Lemma 6.10. For y e A, we have 

P\-^^y\s^y,SSy,S,^y]<2-°-^^'‘. 

Proof. Let be obtained from O by the following operation. 
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• Select X e ® such that <Tj: = Oy and (denoting hy fljc the Z^-th clause adjacent to x) A^^^^®(x, ax) ay) 

at random. 

• Replace the pair of edges {(y, ay), (x, ax)} by the pair of edges {(y, ax),(x, ay)]. 

Let S be the event that ®' satisfies S>. We observe that P [@] = 1 - o„(l) and P [5‘] = 1 - Conditionned on 3>,S 
and ^y, ® and ®' are identically distributed. Moreover, we have 

Corei /2 (®) \ Si /2 (®') c Corel (O') \ Si (O'). 

It follows that 


¥>\-^^y\s^y,SSy,&,^y]=V>\-^^y\s^y,my,3l,S,^y][\ + 0„{l)) 

< Y. P\^^'^^^^iy,ay)=T'\s^y,my,2),S,^y] 

p[xCCorei/2(0)\Si/2(0)|£i§;„@,<g’,'i^y, A^^^’Ofx, = T']. (6.6) 


We define, for T' e r(r') e by T[l'y] - A*^^*0(ay —► y) and r[Z] = r'[Z] for /' ^ I'y (where r[Z] denotes 

the Z-th subtree pending on T’s root). It follows from the same argument as previously for T' e 3"^^ we have 
Py^^j!)j^^[Zy](l) > 1Z2. Therefore 


P [ A'^^’0(y, fly) = r'I .afy, %, t^y 


P[A(4^)0(y,fly) = r\s^y,'Sy]^f^^,]hlyKl) 

Xr'eiT/, P [ A(4flo(y, ay) = T"\s^y,'^y] [/^](i) 


(l + o„(l)) 


^ P [ A'4^)0(y, fly) = T'ls^y.'^y] [Zy] (1) 

“ i:y»e3-^+P[A(4«0(y,fly) = r’ls^y.^Sy] 1/2 
<2P[A'^^’0(y,ay)= T'\s^y,‘Syj, 


Moreover 


P[A'4^^0(y,fly)= r'|.5/y,'i^y] < 

< P [ 0(y, fly) = r'] P [.a/y] ■ V [^^y ] "^ 

<2p[A'^^^0(y,fly) = r'] =2p[A''‘^’0(y,fly) = r']. 

where we used Lemma l6.10l to obtain the second inequality. Using Baye’s theorem once more, we have 


p[AO''‘^^(y,fly)= T' 


P[At4^)0(y,fly)^r'|ggy]P[%] 
P[%|AO(4^)(y,ay) = r] 


A'^^^0(y,fly) = r' % 


In order to deduce the last inequality, we used that by an argument similar to Lemma |6?^ P [ S§y| fly)] > 

1Z2. It follows by replacing in 16.61 that 


P[-.'^y|.!4,%,2>,'i^y] <6 Y P[A‘^^^0(y,fly)= T'15§y 1 P [X C Corci /2 (o) \ Si /2 (o) I®A^^^’O(x, flj = T' 

T'c;^+ 

< 6P [x C Corei /2 (O) \ S 1/2 (O) mx,&,S] 

< 6P [x C Corei /2 (O) \ S 1/2 (0)]P [2>]-ip [(g-]"! 

< 8P [x C Corei /2 (O) \ S 1/2 (O)]. 


We used Lemma lOl to deduce the last inequality. Along with ProDosition l6.4l this ends the proof of the lemma. □ 


Proof of Provosition \5.2\ We take a path similar to the proof of Pronosition lS. 1 1 Let £ > 0 be fixed. Let 

F= |{xE V, A^^^'O(x) is not (£,2/)-cold in 0}|, 

and let afn) be a slowly diverging function. We are going to show that there is a sequence y/ = (1) such that 

E[F(F-1)...(F-q:+1)] < (yrn]“. 
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(6.7) 







This bound implies the assertion as previously. As before, we observe that 

E [F(F - 1)... (F - q: + 1)] < [Ti,..., Ta are not (e,2a-cold], 

where T\,...,Ta are 2£- neighborhoods of a random vertices xi,...,Xa of F. Let ® be the event that jci,..., are 
at distance greater than from each others and have tree-like 5^ neighborhoods, and let A = dTi u • • ■ U dTa- By 
combining Lemma IgTHI Lemma|6j9] Lemma lB.lOl we obtain that for y e A 

In particular, using CorolIarv l6.2l we can apply the result of Prooosition l3.2l to obtain that, for 1 < i < a, 

P [Ti is not (e,2^)-cold|2), Fi,..., r,-_i, r„] < (6.8) 

We have 


P [Ti,..., Fa are not (£,2^)-cold|®] < P [Fi is not (£,2^)-cold|®] P [F 2 is not (£,2^)-cold|Fi is not (£,2^)-cold, ... 

P [Fa is not (£,2/)-cold| Fi,..., Fa -1 is not (£,2^)-cold, 3)]. 


Using <6.81 yields 


P[Fi,...,Fa are not (£,2a-cold|@i] < (o^(l))“. 

Along with the observation that P [3] - 1 - o„ (1), this concludes the proof of the proposition. 


□ 


6.2. Proof of ProDosition l6.41 In order to prove Proposition 16.41 we wiU identify a set of simpler events that will 
imply the proposition. We will first need to control the number of vertices with unusual 2-neighborhood. To this 
end, we let for a formula <1>, Uq be the set of variables such that \ {a e di,ox}| < \d-ifix\ > 2 or such that there 

exists 1< l< k such that |{d_i /x}| > / Z!. Our first condition will ensure that Uo is not too large: 

We now turn to expansion properties of <!'. We define, for a set F c F the sets 

FoiT) = {aeF,\dia\ = l,diflc r,|d_i(fl)n FI > 1}, 

FiiT) = {aeF,d-ianT^0,\dia\ = Z,|difln FI > 1/4} (for 1 < / < k). 

The following conditions encompass bounds on the sizes of the sets Ft (F) when T has moderate size. 

There is no set F c F of size [FI e and such that |Fo(r)| > fc^^^|r|/100. 

For each 1< I < k, there is no set F c F of size | F| e 

and such that |F/(F)| > fc^^'^IFl/dOOZ^). 

Lemma 6.11. Assume t/iat<^ satisfies Then it satisfies Core and Sticky. 

Proof. Let 'P be such that it satisfies l‘Tf0l - <‘^2l . We first prove that ® does not admit a 1/2-sticky set S with |S| e 
j2-o.97t:^ jjy^jeed, let Sc F be a 1/2-sticky set for •P and let 

So = {xE S, |{flE di,ox,d(a,x) n S ^ 0}| > fc^^'‘/2}, 

S/ = {x E S, I {ae d_i /X, ||di(fl,x)| = /, |di [a,x) n S| > //4}| > k^^'^/2} (for 1 < / < k- 1). 

We first observe that 

|Fo(S)| > I So 1/2, and that for 1 < / < fc - 1 |F/(S)| > fc^^'‘|S;|/(2/). (6.9) 

Because S is 1/2-sticky, we have S c SoUuJij S/ and therefore either |Si| > 1S|/100 or there isl</<fc-l such that 
|S/| > |S|/(100/^). In either case, it follows from 16.91 and <^ll -l l^2t that |S/| t Using that (for 

0 < / < fc- 1) |S/| < |S| < fc|S;| shows that Shas size outside the range 

We now turn to the study of the 1 /2-core of ®. Given <P, we consider the following whitening process. Let U -Uq 
initially. While there is a variable x0U such that one of the following conditions occurs, add x to U. 

(a) |{flEdi,ox,|d_i,oan [7| > 1}| > k^'‘^l2. 

(b) |{flEd_ix,|difln U\ > |dia|/4}| > k^''^l2. 
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It is easily seen that the process converges. Let Uoo be the resulting subset of V, then we have 

Core($)i/2=y\t/oc. (6.10) 

We are going to show that Uoo cannot be too large. By condition ISg’OI , we can assume that | LIqI ^ Assume 

for contradiction that \ Uoo\ 5 and let U be the set obtained when precisely - |[7ol variables have 

been added to Uq. By construction each variable xeU has one of the following properties. 

(00) X belongs to Uq, 

(0) X belongs to more than clauses a with dia= {x} and |d_ian (/| > 1, 

(/) xbelongs to more than fc^^^/2 clauses a e d_i_/x with \diar\ U\ > 114. 

Let L/fl c 17 be the set of variables xeU that satisfy (00), Lo c 17 be the set of variables x e 17 that satisfy (0), and 
for l</<A;-lV/cl7be the set of variables x e 17 that satisfy (/). As |17| < |17o| + | Vb| + and |17o| < |17|/fc, 

either |Vbl S |17|/100 > or there is I such that |1//| > |17|/(100/^) > Either case is impossible by a 

similar reasonning as previously and we obtaind that| 17ool < w.h.p.. □ 


Studying ® will be enough to obtain the information needed about ®. Indeed, we shall obtain sufficiently 
strong estimates of the probability of events under the random formula ® to transfer them into high probability 
statements for the biased distribution generating ®. More precisely, say that ® satisfies a property with very 
high probability {w.v.h.p.) iff (13*) has probability larger than 1 - exp under ®. Then we can infer that 

has a large probability under the random formula ®. 

Lemma 6.12. Letsi be an event. Assume that satisfies w.v.h.p.. Then satisfiesw.h.p.. 

Proof. Without loss of generality we can assume that sz/ contains the event 

{all but of the 2£ neighborhood of variables x e consists of a pure tree}. 

Reformulating the definition of the planted replica model, we see that 

Xd) 1 [® C .e/] IP [® = ®] exp(nB(i,_/) 




XcflP [® = ®] exp{nB(S,^() 
sup<i,e^exp(nBd,/) 




Xd) IP [® = ®] exp(nBd),f) 

)r all pure trees T 
sup exp(niJd>,f) s exp |n77(p^^') j < exp . 


( 6 . 11 ) 


We observe that Bd),f - Moreover, for all pure trees T H{pj^] <4 It therefore follows that 


d>ejj/ 


Returning to the definition of (fc, d, f) in Section l42] we obtain on the other hand 

[® = ®] exp(niJd>,f) - l/2exp|n5S*^'(A;, d,/l)j > exp , 

where the last estimate follows from an analysis similar as previsously. Replacing in 16.111 yields 
P [-..c/] < 2exp P [-.^/] < 2exp [2^-^-^^‘^^n - = o„(l), 

as desired. □ 


In order to obtain our result, we are thus left with proving the following proposition. 

Proposition6.13. W.v.h.p. ® satisfies OD . 7D and 1^20 . 

Proof of Provosition \6.4\ The propositions follow from combining Lemma [6.111 combined with Lemma [6.121 and 
Proposition l6.13l □ 
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6.3. Proof of ProDosition ie. 1 31 In this section we shall study typical properties of the random formula ®. For a 
formula ® and 0 < / < fc, we let md®) count the number of clauses a of ® such that |dia| = 1. 


Lemma6.14. W.v.h.p.<b satisfies K(X> . 


Proof. Let Yi denote the number of variables x e V such that \{a e di,oJc}| ^ Let p denote the probability 

that a binomial of parameters q) takes values 0. Using Lemma lSdI and recalling the definition of ^ in 

Section!!] gives 


ik' 


EIFi]< E 

r=0 


''d/2'' 


p\\-p) 


dl2-r 


A simple computation reveals that p-2^ + 6fc(4 ^). This implies that the summand is maximal for r = 4*^^^^ and 

allows to bound E[Fi] as 


E[Fi] < 


fce(ln2) 

fc7/8 


2-*^ + 6fc(4-^) = 6fc(2"*^). 


A standard concentration argument then yields that Fi < w.v.h.p.. 

Similarly, let Fa denote the number of variables xe V such that d_ix > 2. Let Q denote the probability that a 
binomial of parameter {dl2,exp(- I {1- cpq^~^) takes a value larger than 2. By another simple computation, 

we find Q = Oj;(2“*^). It follows from Lemma [6U] that ElFal - Q - dk{2~^). Again, by concentration this implies 
Fa =<2-°'^'^*'w.v.h.p.. 

Finally, for 1 < / < fc let Faf/) be the number of variables xeV with \diii\ > ll\. By similar computations, 
we obtain 


E[F3(/)] < 


d/2 

E 

= fc'+3//! 


d/2 

r 


fc-i 


^(l + 6fc(2-*^)) = 6fc(2-*^). 


It follows that ElFa] = 0^(2 ^), and by the same concentration argument as previously F3 < 2 w.v.h.p.. 

The proof of the lemma is completed by noting that lUolsFi + Fa + Fa. 


□ 


We define mj = ^ ^ fc'"*'^ / /!. The previous estimates can easily be (slightly extended and) recast as follows. 
Remark 6.15. W.v.h.p. we have for all 0 < / < fc, mi{Q>) < m'^. 


We are now ready to complete 
Lemma6.16. W.v.h.p. ® satisfies JD . 


Proof. Given ®, let Xo{t, r, y) count the number of sets T cV of size | T\ = tn, such that 
. IFomi^rtn. 

• Ilae.Fo{T)\d-ianT\ = yrtkn. 

By definition of FifT), Xo(t, r,y) = 0ify< fc"'^. The expected value ofXo(f,r,y) can be computed in the following 
manner. First choose the sets T and FoiT). The latter has to be chosen among the mo(®) satisfied clauses. Among 
the tdn literal clones from T, choose the rtn positive literal clones that will be connected to the positive literal 
clones of clauses inFo(r). and the ytdnfc literal clones that will be connected to negative literal clones of clauses in 
FoiT). Make the same choices among the negative and positive literal clones of the clauses in FoiT). Then match 
these rtn positive literal clones (resp. yrtkn negative literal clones) at random, and then match the remaining 
dn/2 -rtn remaining positive literal clones (resp. dn/2 -rtn remaining negative literal clones) at random. The 
normalizing factor is the total number of graphs that can be obtained from the configuration model, {dn/2)f. 
Without words, and using in addtion Remark [6.15l to observe that we can assume mo (®) < m'^- ^ fc^, this gives 


E[Xo(t,r)]< 


< 


n 

'm'o 

tdn 

tdn 

rtkn 

rtkn 

tn^ 

rtn^ 

rtn ^ 

yrtkn^ 

rtn 

yrtkn^ 

n 

'm'' 

tdn 

tdn 

'rtkn 

' rtkn ' 

tn^ 

rtn^ 

rtn ^ 

yrtkn^ 

rtn ^ 

yrtkn^ 


(rtnk)\(dni2- rtnk)\{dnl2)\ 
{dnl2Yf 


dn/2 

-1 

' dn/2 ' 

rtn 


yrtkn^ 


-1 
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( 6 . 12 ) 


We shall bound this quantity by using the bounds, for l< a<b and n>0 


Mnn<iln 

i b) n 


ybnj 


< bln 


I ae\ 

I b I 


This yields 

1 ( dk^ 1 r 7 \ 12e^t\ 

— lnE[Xo(t, r,y]] < + rtln ^—J + r fln( 2 A:e fj + yrtkln]^ -J. 

In particular for r > t e and y > 1/fc, we get 

-lnE[Xo(f,r,y)] < -flnf+ r + k^'^t\n{k^^t] < 
n 

In particular 

E EE E[Xo(r,r,y)]<exp[-2-0-3«5*^u|. 

fe|2-'>«,2-™lre[fc3"‘,dl ye[0,ll 
fneN rfneN yrtneN 

This implies by Markov’s inequality that w.v.h.p. there are no sets T of size |r| e such that 

\Fom\>k^>*\n □ 


Lemma6.17. W.v.h.p. satisfies P^2D . 


Proof. Given ® and 1 < / < fc, let Xfit, r,x,y) count the number of sets T c V of size |r| = tn and such that the 
following condition are true. 

. \Fim\ = rtn. 

• ZfleF,(r)|5ianr| = xrrfcu. 

• Ila€FiiT)\d-ianT\ = yrtkn. 

By definition of FfiT], Xfit, r,x,y) = 0 if x < lk~^ 14 or y < k~^. With Remark l6.15l we can assume 

mfi<S>) < mj. 


Reasoning as before, we obtain 

E[XKt,r,x,y)]< 

Taking logarithm and using 16.121 . we obtain 

— lnE[X/(f,r,x,y)] < flnf-l + rtln 
n i t) 


'n 


tdn 

tdn 

' rtkn ' 

rtkn 

dnl2 

-1 

' dnl2 ' 

tn^ 

rtn 

xrtkn 

yrtkn^ 

xrtkn 

yrtkn^ 

xrtnk 


yrtkn^ 


-1 


dk‘^^ 


\2^l\rt] 


+ xr tfcln 


2e^t\ {2e^t 

- + yr ffcln - 

X ) I y 

,-i 


In particular, for r > A;^^^/(100Z^), t E [2 and x >/fc ^I4,y>k we have 

1 (e\ (dk^'^^ 

— lnE[Xdt,r,x,y)] < + f fin 


2^rll\ , 


fc3/4 

< -tlnt+ f+ _ fin I 


100/2 




For any l< I < kwe have < k^^ and we thereby obtain that 

This entails that, for any l< I < k, 


-lnE[JC/(f,r,x,y)] < 
n 


E E E E E[^/(f.h^.y)] £exp[-2"° ®®®*'u]. 

jg.[ 2 -o. 98 (:_ 2 -*:/ 20 ],.£|j- 3 / 4 /(ioo/ 2 ),rf] j:e|0,l] ye[0,l| 
fneN rtneN xrtn&iyrtn&f>i 

This implies by Markov’s inequality that w.v.h.p. there are no 1 < / < A; and no sets T of size 1 T\ e [2~ 
such that |F/(r)| > fc®'4|r|/(100/2). 


0.98fc^,2-fc/20„] 

□ 


7. Moment computations 

In this section we prove Lemm df2J\ Lemma [2^ and Provosition \4.5l We recall that q- q{k, d, p) was defined in 
Section\^ Eq. O.li . 
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7.1. Preliminaries. We will need the following version of the inverse function theorem. 

Lemma 7.1. Let t/c bean open set and let fe C^{U]. Assume that ueU and r > 0 are such that 

{xE : ||x- u \\2 <r}cU. 

Let Df{x) be the Jacobian matrix off at x, id the identity matrix, and || • || the operator norm over L^ (R^). Assume 
that Df{u) = id and 

\\Dfix] - id|| < - for all x e such that \\x- u ||2 < r. 

Then for each y e R^ such that ||y- f{u)\\ < r/2 there is precisely one x e R^ such that ||x - u|| < r and /(x) = y. 
Furthermore, the inverse map f~^ is C“ on {x e R^ : || x - u|| < r], and Df~^ (x) = (D/(x))“^ on this set. 

We will also need the following result on the large deviation function of the multinomial distribution. 

Lemma 7.2. Let Z > 2 and {pi,..., pi) e (0,1)^ satisfying pj - 1 be fixed. We have, for any lq\,...,qi) e (0,1)^ 

satisfyingY}j^i qj - 1 

— InP [V j E [/], |Multinomial(n,pi,...,p/)y - nqj\ < 0.01\/n] = ^ ^ 7 jln^—1 + o„(l). 

^ j=i V j 


Finally, we will need the following concentration result. 

Lemma 7.3. Let d < d^-sAT tmd /3 e R be fixed. For any a > 0 there is6>0 such that 

< exp(-dn), 


-lnZ®(;6)--Eln[Z®(;6)l 
n n 


> a 


< exp(-5n). 


Proof. The proof follows from the fact that if two formula O' differ hy at most one switch of edges, the associated 
partition functions satisfy 

|lnZ(i)(;6) -lnZj,/(/3)| < 2/3 and, for ct e {-1,1}” \ < 2/3. 

The stated concentration result is then a consequence of Azuma’s inequality (applied to the configuration model). 

□ 


7.2. The first moment computation. Let : R x (0,1) ^ (0,1] he defined hy 

■Zl,J:()S,h) = l-C/3(l-Zz)^, 

the Kullhack-Leihler divergence Di: (0,1)^ ^ R be defined by 

Di{a,h) = ’ 

and fi^k : R^ ^ R be defined by 


/i,fc(rf,;S) = ln2+ -lnzi,i;(;S,l-< 7 ) + £ZDi 
k 



Proof of Provosition \4.5\ We need to compute the expected value of OaeF Va,p (o') under a random assignment cr e 
{-1,1}”. To do this, we introduce a different probability space formed of all vectors in {-1,1}*^"' x {0,1}"* 


ae|m|,;e[fc| i (ya)ae[m] 

with a probability distribution P such that the {(pal)a&[m]M[k] are independent random variables distributed as 
= 1) = l-qandthe (ya)ae[m] are independent Bernoulli random variables ofparameterexp(-/3)/(l+exp(-;6)). 
We consider the two events 


and 


S = {V flE [m] (ya = 0and3 / e [k],(pal = 1) or (ya = 1 and V Ze [k],(pal = -l)} 




\{(a, l):(pal = 1)1“ 
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Then, using that y/a,pifl>al) = (l + exp(-/3)) (P [y^ = O] Ig ieik],<Pai=i + ^ [Ta = O] Iv ie[fci,we see the expected 
value ofYla€FVa,p under any given assignment n e {-1,1}” is given by P[S|ii](l + exp(-/l))'”. In particular, 


1 , , 1 d 

— InE [Z/3(®)1 ~ ln2H— lnP[S|iJ] H— ln(l + exp(-d)). 
n ^ n k 


By Bayes’ theorem we have 

It follows from Lemma lT^ that 


P[S|B] = 


P[S]P[B|S] 

¥[ 8 ] 


1 


^lnP[B]~-Di|-,l-^?|. 


(7.1) 


(7.2) 

(7.3) 


It is also straightforward to obtain that 

— lnP[S] ~ Zi,fc(;S, 1 - < 7 ) -ln(l + exp(-)S)), 
m 

and by definition of q we have (using the central limit theorem) 

— lnP[B|S] = o„(l). 
m 

The proposition is obtained by combining Ea. l7.1H73) . 


(7.4) 


(7.5) 

□ 


7.3. The second moment computation. Recall that cp - \ - exp(-/3). Let - {{h,h) e (0,1)^, h < h}. Let Z 2 ,fc ^ 
IR X ^ ( 0 , 1 ] be defined by 

Z2,j;(/3, h,h) - I- 2cp{l - h)^ + cjjil - 2h + h)^. 

Lemma 7.4. Let g 2 ,k,p be defined by 

h+(h-h)[l-cp(l-h 

Z2,kiP,h,h) ’ Z2,k{P,h,h)^ 

Let a E (0,1) and let^ - {(x,y) e P^, || (x,y) - (i, ^ k2~^]. Then the equation g 2 ,k,pih-, h) - (i, i^) admits a 

unique solution in3~ that we denote by {h^{a), hp{a)). Moreover, a hp{a) (resp. a hp{a]) is of class C“ on 

( 0 , 1 ) and the following is true. 

( 1 / 2 ) = ( 1 / 2)2 = 

h'p{a)^h'p{a)-\l2+dk{2-^^'^) for\a-\l2\<2-^'^. (7.7) 

Proof The Jacobian matrix Dg 2 xpih,h) of g 2 ,k,p at {h,h] e is given by Dg 2 ,k,^{h,h) = id+ djc{2~^); in par¬ 
ticular it satisfies \\Dg 2 ,k,pih,h) - id|| < 1/3. Then Lemma [TTI applied to g 2 ,k,p ^ C°°(/J') with y - u - 
and r - k2~^ imply that there is exactly one {hp{a),hp{a)] e such that || (h^(a:), h^gja)) - II 2 ^ r and 

g 2 ,k,pih^[a) ,hp{a)) - (i, Moreover, the map a [hp[a),hp[a)) is of class C“ and {h'^ia], h'^ia]) = (0, -1/2) - 1 - 

Ok{2~^). A more detailled computation (using ■j^g 2 ,k,pih^{a), hp{a)) - (0,-l/2) and the chain rule for computing 
derivatives) reveals that, for In- 1 / 2 | < 2~^^^ 

h'p(a] + ^Z2,kiP,hp{a),hp[a)) = ^ (lS,^/3(a).^/3(a)) = Ofc(4"*^), 

h'pia) + cph'p(a]2^~'^ + dk{2~‘^’‘'^) = ^ |h^(o:)^||^ - 1 -h^(a)^^| {f,hp[a],hp{a)) = Ofc(4"*^). 

In particular 

h'^ia) - h'^ia) + ^ ^ (1 - Z 2 ,k{f, hfi{a),hp{a]]] - 2-^cp -t dk{2-‘^^'^) = 6k{2-^'^'^). 

Finally, 17.61 is easily proved by inspection. □ 


g2,k,p(h,h) = 


In particular, we observe that Proposition l4.5l and the above lemma imply the following. 
Corollary 7.5. We havelimn^oo - |/2_j;(d,/5,1/2). 
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Let the Kullback-Leibler divergence Dz : (0,1)^ ^ IR be defined by 


, - .fa] 1-a, fl-a] l-n, f l-a i 

Dz(a, h, h) = aln -^ H-In —^ H-In -^ , 

\2[h-h)l 2 \ 2h ) 2 \2[l-2h+h)l 


and / 2 ,fc : IR^ x (0,1) ^ IR be defined by 


fz.kid, p,a) ^\n2 +H{a) + - In Z 2 ,k iP, hp [a),hp (a)) + dDz [a, hp {a],hp (a)). 

fc 


We also let 


so that 


Z[d,p,a) = E 


E Uiv^ a,pi.(^a)Va.piJa)] 

aeF 

a-r={2a-l]n 


e[z2(0)1= Y. Z{d,p,a]. 


ae{0,l/n,...,l) 

Proposition 7.6. Let d>0,peM. and I c [0,1] be fixed. We have 


-In 

n 


and in particular 


Y Z{d,p,a)\~ sup fz,kid,p,a) 
VaE{0,l/n,...,lln/ / 


-lnE[z?(®)]~ sup fz,k{d,p,a). 
n 1 P j aeto,!) 




Proof. We need to compute the expected value of under a random pair of assignements 

e {-1,1}^”. To do this, we introduce a different probability space formed of aU vectors in {-1, x {0, l}^'" 

(^al)a€[m],l€lkl> (ya^ > y^^la£[ml 

with a probability distribution P such that the (<pal)a€im],l£ik] independent random variables satisfying 

(1,1), with probability(a) 

(1,-1), with probability(a) - h^ia) 

(-1,1), with probability h^ia) - h^ia) 

(-1,-1), with probability 1 - 2hpia) + h^ia) 

independently for all a, I, and the iya^aeim] (resp. (yi^bae[m|) are independent Bernoulli random variables of 
parameter exp(-/3)/(l + exp(-;6)). We consider the following events. 

Si = {V flE [m] (y™ =0 and 3 I e [k],cl)ai e {(1,1), (1,-1)}) 

or (y^i’ = 1 and V / e [A:],(/)^/ e {(-1,1), (-1,-1)})}, 

Sz = {V fl E [m] (yf = 0 and 3 / E [k],cl)ai =e {(1,1), (-1,1)}) 

or (yf = 1 and V / E [fc],(/)^i e {(1,-1), (-1,-1)})}, 

S'2^ = SinS2, 


and 




|{(fl,/);(/)«, = (l,l)}|-i^zz 


<^, \\{ia,l]:cPai=i-l,l)]\-^n\<^ 
\/n I 2 I y/zz 


\\{ia,l]:cPai = i-l,l)]\-^n\<^, 
I 2 I i/zz 


\{ia,l)-.(Pat = i-l,-l)}\-^-^n 


\/n 


Then the expected value of Oaef Va,p ftlVa.p M under any given pair of assignments (cr, t) e {-1, l}'^” that satisfies 


|{Z E [zz] : ((T;,T;) = (1,1)}|- E^ZZ 


< v/zz, ||{Z E [zz] ; (f7;,T;) = (1,-1)}|- -zz| < v/zz 


||{z E [zz]: ((7,,T;) = (-1,1)}| - -zz| < '/n, 


|{z E [zz]: (fT,-,T,) = (-1,-1)}] - ^-^ZZ 


< \/zz. 
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(7.8) 


is given as previously by IP[S^^^ (1 + exp(-/3))^'”. In particular, 


-InEfz^C®)! ~ln2+H(a) +- sup lnP[B'^’|S'^’] + ^ln(l + exp(-;S)). 
n 1 p J «ae(0,i) k 


By Bayes’ theorem we have 


P[5(2)|^(2)] 


P[S^2)]P[b12)|5(2)] 

P[B(2)] 


It follows from Lemma lT^ that 


-^lnP[B‘^^] ~ -D2{a,hp{a),hp{a)). 
Icjn 


It is also straightforward to obtain that 


■^lnP[S'^^] ~ Z2,ki.P,hp{a),hp{a))-2\n{\ + ex:p{-^)), 
and by definition of we have 

— lnP[B‘2’|S‘^>] = o„(l). 
m 

The proposition is obtained by combining Ea. l7.8H7.12l . 

Lemma 7.7. Assume that d < d^-sAT t^tid e K. Then we have 

sup f 2 ,k(d,p,a]< f 2 ,kid,P,'^/ 2 ] 
a>2-'=/io 


(7.9) 

(7.10) 


(7.11) 

(7.12) 
□ 


Lemma 7.8. Assume thatd < d-[k) or thatd e [d-{k),dk-sATl tind thatp< p-{k).Then we have 

sup f 2 ,k{d,p, a) < f 2 ,kid, 13,112] 
ae(0,l) 

We defer the proof of these lemma to Section [731 

Proof of Lemm d2J] The proposition follows by combining Corollarv l7.5l Proposition l7.6l and Lemma [7(^ □ 

7.4. Proof of Lemma l2.21 To facilitate the proof of Lemma|2j2]we introduce a random variable that explicitly con¬ 
trols the “cluster size” ifi). More precisely, we call cr e {-1,1}“ tame in <1> iff 

^^,aiP)<E[Zfim]. 

Now, let 

Ztame(®i P) — E n (O'a)lo' is tame- 

(re{-l,li"aeF 

We shall also need to introduce a few more notations: we denote hy m- [mo,..., mk) a vector of [m\^^^, and by 
m(®) = (mo(<!>),..., mj;(<!>)), with the myT®) as defined in Section [Q] Also recall that a denotes the all 1 vector of 
length n. 

Lemma 7.9. Let d < dk-sAT titid peRbe fixed. Assume that 

P [a isa tame in ®] > exp(o„(n)). 

Then 

E[Ztame(®,iS] > exp(o„(n))E [Z/ 3 (®)] . 

Proof. Given that m(®) = m(®) the two formula ® and ® are identically distributed. Thus we have for any m e 

[y^]fc+i 

PIct is not a tame in ®|m(®) = m] = PIct is not a tame in ®|m(®) = m]. 

In particular this implies that 

E[Z^(®)-Ztame(®,i8)]=2” ^ P[ & is not a tame in ® | m(®) = m] P [m(®) = m] exp(-j8mo) 

me[m] 

< 2” ^ P [d- is not a tame in ® | m (®) = m] P [ m (®) = m] E [ Z^ (®) ] 

me[m] 

< IP [a is not a a tame in <S>] E [Zjg(O)]. 
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This concludes the proof of the lemma. 

Lemma 7.10. Assume that d < dk-sAT tmd peU are such that 

E[Ztame(®,J0)] 


Then 


Proof We let 


E[Z^(<D)] 

E[Ztame(«l»,)8)]" 

E[Ztame(®,;6)2] 


>exp(o„(n)). 


> exp(o„(n)). 


Ztameid, p, OC] — E 


E n iVa.p [(^a)Va,^ (Ta)) 1 ffis a tame 1 ris a tame 


a€F 

a-T=i2a-l)n 


Then we have, by the definition of a “tame” 

-^tameC^j ^) — E 

q;< 2 -A :/10 


E n atame‘^Ai,o-ip) 

ael-l,!}" aeF 


On the other hand we have with Lemma lTTT] 


This implies that 


y Ztsmeid,p,a)< y Z(d,;S,Q:) = exp(o„(n))E[Z^(®)]^. 

Q.>2-fc/io 


]“ -^tame(^t .Ztame(^j 


a<2 


a>2 


= exp(o„(n))o(E[Z^(®)]^j. 

The lemma then follows from the assumption that E [Z^gfO)] < exp(o,i(n)) (E [Ztame(®.i8)])- 
The reverse direction of Lemma l2.2l wiU be given by the following lemma. 

Lemma 7.11. Assume that d e [d_ {k],djc-sAT] <^tid P> P- (fc) are such that (2.3V holds. Then 

-Eln[Z^(®)]~-lnE[Z^(®)]. 


Proof. We can apply Lemma[7j^to find that 


E[Ztame(q»,l3)] 

E[Z^(®)] 


>exp(o„(n)). 


Hence Lemma [7.10l implies that 


. ,E[Z,ame(®,;S)]2 

limint^rT^:-—^-TT- > exp(o„(n)). 


E[Ztame(®,;S)^] 

Using the Paley-Zigmund inequality we have 

liminfP [Ztamei^,P) > E[Ztame(®,;6)] 12] > exp(o„(?l)). 

In particular, we have 

lmmfp|Z/3(®)> yE[Z^(®)]] >exp(o„(n)). 

In other words 

11 1 . . 

> exp(o„(n)). 


liminfP 

n—*oo 


- InZfll®) > - InE [Zfi(®)] - o„(l) 
n ^ n ^ 


It follows from Lemma|7j3]that 


iEln[Z«(®)] > -Eln[Z«(®)] -o„(l). 
n ^ n r i 


□ 


□ 
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The proof is completed by Jensen’s inequality which give us 

-Eln[Z^(®)]<-lnE[Z^(®)]. 
n n 


□ 


The second part of the proposition will be a simple application of the following lemma, which is similar to 
Lemma rTTg] 

Lemma 7.12. Letd < dk-sAT ‘^nd fieU be fixed. Assume that there exists a sequence of event Sn such that 

IP [® e 5'„1 = 1 - o„(l) and liminfP [® e < 1. 

Then we have 

lim inf — In E [ Z^ (®) ] < lim sup — E In [ Z^g (®) ]. 

n—*oo fi n—*<yD n 


Proof. Let m{n) be a monoticaUy increasing sequence and ^ be suchthat P [® e < exp(-f m(n)). We have, 

by the same steps as in the proof of Lemma[7j9] 

E ] < P [® e §m{n)]^ [Zp{<D)] < exp[-^m{n))E [Z(®(m(n), k,d),p)]. 

Thereby we obtain, using that ^ In Z^ (®) e [- ;6 + In 2, In 2] for all ® 


1 

m{n) 


Eln[Z^(®)] = 


< 


< 


-Eln [Zfi(®)la,pS l+o„(l) 

——InE [Zfl(®)la,pS l+o,j(l) 

^Eln[Z^(O)]-f+0„(l). 
m{ri) ^ 


□ 


Proof of Lemm d2^ Assume that Eq. 12.31 holds. Then by Lemma [7.11l we have -^Eln [Z^gC®)] ~ ^ InE [Z^(®)]. 
Assume that Eq. Il2.3> does not hold and let e be such that 

lim sup — E [lnS#j 5 liminf — InE [Z^(®)] + e 

Letz= -^InE [Z^(®)]+e/2 and be the event that-^InZ^l®) > ■^lnE[Zy3(®)]+e/2. Then using Jensen’s inequality 
and Lemma [7l3] we obtain P [® e Sn\ ~ J while liminf„^oo P [® e < 1. Therefore with Lemma|7T2]we have 

liminf— EIn [Zjg(®)] <limsup —lnE[Z^(®)]. 

n—*(X) fi j2—*oo Tl 

□ 


7.5. Proof of Lemma lTTTl and Lemma lTlSl We first need to study/ 2 ,fc(d,/l, ■) locally around a = 1/2 and compare it 
with fi_icid, fi)■ This will be given by the two following lemmas. 

Lemma 7.13. We have, for d < dk-sAT cmd fieM., 

/i,fc(d, fi) = ln2- ^ (c/i2-*^ + 2-1-2*^ - 

Proof. The result follows from a direct computation, using the observation that q-j + Cp2~^~^ + 0^(4“*^). □ 

Lemma 7.14. Letd < d]c-sATtindfie P be fixed. / 2 _fc is of class C°° onM.^ x (0,1). It satisfies fz^jfid, fi, 1/2) = 2fiic[d,fi), 
■tf2,kid,fi,ll2) = 0fludsup|„_j/2|<2-*:/3 ■£if2,kid,fi,a) < 0. 
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Proof, a H{a) is clearly of class C“ on (0,1). Similarly, (/3, h, h) \nz 2 ,kiP, h, h) is of class C“ on and D is 
of class C°° on (0,1) x The smoothness of a —► fi.kidyf, a) therefore follows from the one ofa^ (/z^(o:), h^gla)) 
granted hy Lemma FTTl 

Because (hp{a),hp(a)) satisfy g 2 ,j;,/ 3 (h^(a), h^(o:)) = (i, we have using the chain rule 

^^^{f,hp[a),hp{a)) = ^^{a,h^{a),hp{a)), 

dh dh 

The differential of / 2 jt with respect to a then simplifies to 

5/2 k , d -- 

- ’—{d,p, a) - H {a) + d — Dzia, hnia], hnia)) 

da da 


= H\a)-dH'{a)+^\n 
Sfzk, 


hi}{a){l-2hi}ia) + hp{a)) 


{hp{a) - hp[a))^ 


In particular, for a-1/2 we have h^(l/2) = hp{l/2)^ and -^id,p, 1/2) = 0. 
Differentiating once more with respect to a yields 


d^fz.k 

d^a 


{d,p,a) - H"{a) + d——rD 2 {a,hp{a),hn{a)] + d——D 2 {a,hp(a],hp(a]) + d - —D 2 (a, h^{a), h^(a)) 

da^ dadh dadh 


= H"[a) - dH" {a) + 


d 

H- 

2 


hp{a)-h^{a) \-2hp{a) + h^{a) 

1 2 


h'p(a) 


hp{a) \-2h^{a) + hp{a) hp{a)-hp{a) 


h'pia). 


In particular for |o:- 1/2| <2 
d^f2.k , 


-kl3 , 


! have with Lemma[7j4] 


'^{d,B, 1/2) = H''{a) - dH"[a) - Sdh'Ja) + 8dh'„[a) + Ok(2~'^'^) 
d^a P ^ 

h'p(a)-h'i^{a) + ^ 


— —4 + 8ci 


+ dk(2-^'^) 


= -4+6fc(2-*^/3). 


□ 


We now study f 2 ,k{d,f,a) when \a- 1/2| > For the sake of readability, we decompose this study in 

small steps. We first show that we can upper hound f 2 ,k by a simpler function. Let Z 2 ,fc : 0? x (0,1) ^ IR and f 2 ,k ■ 
IR^ X (0,1) ^ IR he defined hy 

Z2.kif,a) = l- 2 c/ 32 "*^ + c^|^-y^j , f2,k{d,p,a) = hi2 + H{a)+ jln[z2,kid,p,a')). 

Lemma7.15. For all a e (0,1) we have f 2 ^kid, p,a) < f 2 ^kid,p,a). 

Proof. Consider the event defined in the proof of Pronosition l?. Bl and their prohahility P' under the distri¬ 

bution where (with the notations of the proof of Prooosition l7.6l 

(1,1), with probability (1-a)/2 

(1,-1), with probability a/2 

4>al - 

(-1,1), with probability a/2 
(-1,-1), with probability (1-a)/2 

independently for all a e [m], I elk]. We have 

^lnP'[S‘2^] ~ In |l - 2c^2"*' -I- j j - ln(l-t exp(-)S)), 
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— lnP'[E'2’] ~ 1, 
m 

— ~ — lnP[S'^’|B'2’]. 

m m 

In particular, with Bayes’ theorem 

/2 fc (rf, j8, a) - /2 fc (rf, |6, a) ~ — In P' [S'2’ ] - — In P [S‘2^ ]-3- P'/3 | S'^’ ] > 0 

’ ’ mm m 

□ 


Lemma 7.16. Assume that d < d]^_sAT tind peU. Then 

f 2 ,k[d,l 3 M 2 - < f 2 ,k{d,p,ll 2 ). 

Proof. We compute 

f 2 ,k{d,f,\l 2 - = 21n2- ^ (-Cjg2^"*' - c^2"^*'] + 6fc(2"‘*^'^) 

On the other hand using Lemma [7.14l and Lemma [7.13l we have 

f 2 ,kid,pM 2 ] = 2fi,kid,l3) = 21n2 - 2j (c^2-*^ + 2-^-^'‘ - k2-^-^^'^ + Ofc(4-*^). 

It follows that 

f2,k{d,f, 1/2-1^2-’^'^] - f2.k{d,f, 1/2) < -k‘^2^-^ + dOkiA-'^) + 0^(4"*^) < 0. 

□ 


In order to prove Lemma lTTTliy.SI it will be convenient to restrict the range of {d, p) that we need to consider. We 
define f2^k : P x (0,1) ^ P and Z2,fc : (0,1) ^ P by 

f2,kid,a)= lim [2,kid,f,a), Z2,S:(a) = lim Z2,kif,ot)- 

^—*00 /3—»oo 

The following claim is immediate, once one observes that mhkid,a) = |ln(z2,fc(d,;6,o:)). 

Claim 7.17. Assume that d < dk- sat- Then for a e [0,112-k^2~^^^] wehave 

^f2,kid,cc)>^f2,kid,ll2-k^2-^'^]. 
od od 

Similarly, we have the following. 

Claim 7.18. Assume thatd < dk-sAT- Then fora e (0,1/2- k^ 2 ~^^^] we have 

^f2,kid,f,a] > ^f2,kid,f,\l2-k^2~^'^]. 
dp dp 


Proof. We compute 


In particular 


^ f R , _ exp(-/l) d l-cp[\-aV 

gpfi.k ,P,a + 


d^ - dexpi-B) j. 1 - 1. 

f2,k[d,p,a) =- - a) (1+ Ofc(2-^)) < 0. 


dadp 


□ 


Therefore, in order to prove Lemma lT/Tl we can assume that d = dfc_sAT and f^oo, and to prove Lemma[7j8l we 
can focus on the following two cases. 

• d-d- [k] and ^6 —► 00 , 

• d = dk-sKt and p = f-[k). 
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Lemma 7.19. We have 


sup 

ag|2-A:+10,i/2_J;22-*:/2| 


f2,kid]c-SAT>0C) ^/2,fcWfc-SAr. 1/2-fc^2 


Proof. We first compute 


f2,kidk-SAT, 112 -= -k'^2^-'^ + 6fc(4-*^). 


We differentiate /a.jtidfc-SAXi oi) with respect to a. 

df2.k 


da 


idk-sAT,a) = 

1 - ai 


dk-SAT (1 - «) 


jfc-i 


Assume that a e [1/2- 112-2 Then we have 


fc/3i 


df2.k 

da 


(dfc-SAT.o:) 5 - In 


l/2-fc^2“ 


l /2 + fc 22-*^/2 


Assume that a e [0.4,1/2-2 Then we have 


df2.k 

da 


(dfc-SAT.o:) 5 - In 


1 / 2 - 2 “ 


112 + 2-’^'^ 


2 *^ . 22 ,*: (a) 

(A:ln2)2^"*' (l + 0^(2"^)] > 0. 
(A:ln2)(0.6)*'"^ [l + 0^(2"'^)] > 0. 


Similarly, for a e [2(lnfc)/fc,0.4], we have 

5/2,fc 


da 


0.41 In 2 


(dfc_SAT-«) - — + Ok{{lnk)k ) > 0. 


(7.13) 


(7.14) 


(7.15) 


(7.16) 


For a E [2 ^^^°,2(lnfc)/fc], we compute with the help of 17.131 . and using -(1 - exp(-jc)) < -x/2 for 0 < jc< 1 
f 2 ,k idk-sAT, a) < a |-lna- + Ofc(a) < fz.kidkSAT, 112-k^2~'^'^). 

The lemma follows from Eq. l7.14H7.16l and 17.171 . 


(7.17) 


□ 


Lemma 7.20. We have 

sup f 2 ,kid-{k),a) < f 2 ,kid-{k),l/ 2 - k^2~’‘^^). 

ae(0,l/2-fc22-*:/2| 

Proof. We first compute 

f2,k{d-{k),ll2 - k^2~'"''^) = k^2^~'" + Ok{k'^2~^). 

By Lemma [7. isl and Claim [TTTl we also have 

sup / 2 ,fc(d_(A:),a) < / 2 ,fc(d_(A:), 1/2-A:^2“^'^). 

a:e(2-*no ^/2-fc22-“] 

Let a* he a maximum of / 2 _fc(d_(A:),-) over (0,2“^^'^°).The equation _ q 

-ln(T^^) = fcln2(l + Oi;(l)). 

Expanding for a < we obtain that a* ~ 2“^. Using 17.201 once again yields 

Q:* = 2"i"^ + Ofc(4"^). 

In particular 

f 2 ,k{d-{k), a*) = ln2 + k2~'^ + 2~^ + [-2"^ - - k2~'^^^ + 6k[^~^) 

^k^2-'^ + Ok[A-’^). 

Noting that lima^o ■§^f 2 ,kld-{k),a) - oo and using <7.181 . this gives 

sup f 2 ,k{d-{k),a) < f 2 ,k[d-[k),ll 2 - k^2~^'^). 

ae(0,2-‘:nO) 

Collecting 17.191 and 17.211 ends the proof of the lemma. 


(7.18) 

(7.19) 


(7.20) 


(7.21) 

□ 
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Lemma 7.21. We have 


sup f2,k{dk-sAT,P-i.k), a) < f2,kidk-SAT,P-(k),1/2- k^2 

ae(0,l/2-fc22-*:'2j 

Proof. By combining Lemma [7.19l Claim [7?T71 and Claim [7!T8l we obtain 

sup f2,k(dk-SAi,P-ik),a)^f2,k(dk-SAi,P-ik),l/2-k^2-'‘^^). (7.22) 

ae[ 2 -*:U 0 _i/ 2 _fc 22 -J:/ 2 ] 

Let a* be a maximum of f 2 ,kidk-SAT>P-(k),-) over (0,2“^^^°).The equation = 0 is again given by 

17.201 and hence 

a* = i-2"^"*' + Oj;(4"*'). 

In particular 

f2.kidk-SAJ,P- ik), a*) = ln2 + k2-^ + 2"*^ + - 2-1-2'^ - k2-^^^ + 0^(4-^) 

= Ofc(4-^), 

while 

/2,i:(rffc-SAT,;6-(fc),l/2-fc^2-^'i'’) = fci“2i-^+0fc(A:^2-*^). 

Noting that lima—o -^fz.kid, f- (A:), a) = oo, this gives 

sup f 2 ,k(.dk-sAJ,P-(.k),a) < f 2 ,kidk-SAJ,P-ik),l/ 2 - k^2~’‘^^) 

ae(0,2-*'UO) 

This concludes the proof of the lemma. 

Proof of Lemm diTTA Let d and f be as in Lemma lTTl We first observe that for a e (0,1/2), f 2 ,k{d,f, a) > f 2 ,k{d,f,\- 
a]. Therefore, we can restrict ourselves to a e (0,1/2). By Lemma [7.14l we have 

sup f 2 ,k{d, f, a] < f 2 ,kid, f ,1/2). 

l/2-2-*:«<a<i/2 

Combining Claim [7T7]l7.17l with Lemma [7.19l we obtain 

sup f 2 ,kid,f,a)< f 2 ,kid,p,l/ 2 -k^ 2 ~'‘^^]. 

ae|2-‘:U“,l/2-jt22-‘:'2] 

Using in addition Lemma !?, isl and Lemma [7.16l ends the proof of the lemma. □ 

Prood ofLemmc A 7.81 The proof follows by combining the previous results with Lemma [7.20l and Lemma [7.21l □ 
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